4 Reasons why mobile ad attribution numbers never match up



 

If you develop a mobile app and drive new installs through mobile user acquisition, you probably know how tricky it is to measure the ROI of each ad network. 

Facebook, Instagram, Twitter, Google, they all have their own dashboards with different measurement policies, while you, as the owner of one app, struggle to understand your overall return on ad spend, searching for reliable, accurate and consistent source of attribution numbers. 

 

Our approach: having a single point of truth  

For us at Missbeez (hottest startup according to WIRED), the strategic decision was to bring back the attribution data from the different marketing channels we use to our own system of records.
We have implemented it by using AppsFlyer, a leading SaaS mobile marketing analytics and attribution platform.

Doing so allows us to:

  1. Reduce our dependencies on multiple external tools and dashboards 
  2. Track installs from all marketing channels (including non-ads) in our own consolidated dashboard 
  3. Have the attribution data embedded within our core data model, ready to be used by our core business logic.
  4. Analyze the behaviour of our marketing channels in the long run (weeks, months, even years). 

 

Running into discrepancies   

This process revealed what many mobile marketers already know: the numbers do not match. As written in this blog post by Adjust

 

It’s an issue that arises on every single analytics platform - in one dashboard you have 5005 installs, on another, 7246. What’s the deal?

 

After doing some digging, talking to experts and exploring our Facebook ads, I would like to try and explain (also as a self persuasion), why install numbers as shown by Facebook will probably never match the ones companies see in their own systems. 

 

So there it is:  

4 Reasons why mobile ad attribution numbers never match up

 

 

Reason #1: Facebook counts installations while you count users

If like us, your system keeps track of new users then I guess each user needs to do something before he becomes one. 
Whether it’s adding an email, entering a nickname, or even just skipping the welcome tour - users might drop along the process, causing discrepancies between your data and other systems:  

Some systems count number of installations regardless to opening the app (Apple and Google). According to AppsFlyer, 10% of apps are installed but never opened
Some systems count the installation only once the app launches, regardless to what the user actually does when opening the app (i.e. Facebook). 
Some systems (like our own) only count a user as an actually user once he completes a short verification step. 

So continuing with our own system as an example, in the case of low engaged users that didn’t convert to fully active ones - the numbers might be completely different in each system: 

  • Users that installed the app but never opened it will still be counted by Apple & Google as an install (but not Facebook and our own system).
  • Users that opened the app but didn’t go through the verification step will still be counted by Facebook (but not our own system)
  • Only users that opened the app and finished the verification step would potentially be numbered correctly by all systems. 

 

Now that we understand it, what differences should we expect to see? 

It depends on the drop rate during the validation/registration step, which depends on the level of complexity of this step. 

Accurate benchmarks are hard to find but I did find the following: 

  • 54% install-to-register conversion rate in games (according to LiftOff)
  • 35%-47% install-to-engaged conversion rate (iOS, non-organic users, according to AppsFlyer )

Looking at our own numbers (using MixPanel) it seems like 85%-90% of our new users convert from install to register. 

  • 5%-7% drop during the tutorial (indicating some users didn’t fully understand what the app was about prior to installation) 
  • 5%-7% drop during SMS activation process (which makes sense as some users may not feel comfortable with this validation method but we do need it at this point).  

 

 

 

Reason #2: Facebook shows inflated numbers because some installs are counted more than once 

Ignoring the low engaged users who don’t even finish the verification step - what else can Facebook count that we do not? 

Well, since our system counts ‘users’ and Facebook counts installs - there may be more weird scenarios resulting in Facebook showing inflated numbers: 

  1. Users upgrading their devices (every 21 months) will create additional “install” events (counted by Facebook) without increasing the total “users” count.
  2. Users installing the app on multiple devices will create additional “install” events
  3. Users re-installing the app will trigger extra installation events without increasing the “users” count. 

 

Note: according to Tune - re-installs account for 40% of all installs

  • 18% say they deleted an app because it was buggy, and they reinstalled to force a reset to a better-functioning app
  • 13% say they deleted an app that they later found out they needed … and therefore re-downloaded it.
  • 6% say they deleted an app by mistake, and had to reinstall it.


It might be an exaggerated number (for marketing purposes), but even if we take 10%-20% re-installs it’s still a lot! 

So to summarize: we have device upgrades, multiple devices, re-installs, all increasing the number of installs as reported by Facebook and the likes, without bringing a single user in. How unfortunate… 

 

 

 

Reason #3: Facebook’s attribution time window is much longer than you would expect 

Facebook app installs are logged when one of the following occurs:

  • A person clicks on an ad or link, and installs the app within 28 days
  • A person views an ad or link, and installs the app within 24 hours

(source

Since a smartphones are basically a wild playground for interruptions, users may see your ad, and immediately get diverted by the next item on their feed or by any other incoming thread on their smartphone. Such users might later go back to the AppStore (few hours or few days later) and install your app. 

Such scenarios will be counted by Facebook as installs but depending on your implementation (there are few ways to address this issue) they may not get attributed in your own system. 

The date reported as the 'app install time’ can also be deceiving because (according to Facebook): “The date used for reporting app installs is either the day the ad was last clicked or viewed” which can happen up to 28 days before the app was actually opened. 

If you are measuring your ad performance on a weekly basis - remember that time differences can cause a lot of discrepancies.  

 

 

 

Reason #4: Technical glitches 

Passing back attribution data to your own system is a bit tricky. We do this using AppsFlyer, as AppsFlyer manages the integration with thousands of different ad networks. 

From a technical standpoint, this is not a 100% bullet proof since the API’s are based on remote events (coming from AppsFlyer servers), and by the nature of mobile apps events might get interrupted by real-world scenarios (disconnections, app termination). 

Without getting too technical, I will just say that despite all attempts to solve those challenges, I’m 100% sure there are rare scenarios where the attribution data will not be accepted, resulting in mismatches and inconsistent numbers. 

 

 

More references: 

Here's Why Your Facebook Conversion Numbers Are Wrong

Troubleshooting different number of installs between Facebook Ads Manager and Facebook Analytics

Discrepancies: Why doesn't data always match up?

Discrepancies - How Does AppsFlyer Count Installs Compared To App Store and Google Play?

 

 

If you know of other reasons why the numbers are wrong or if you found a way to normalize them and make them consistent - please share your comment below.  

 

Comments

Anonymous said…
Thank you for sharing
https://aab-edu.net/
Vomail said…
Reading this post has really increased my knowledge about technical advertising. Descriptive reporting submitted by facebook ads services hired by my sister for her boutique promotions is highly presentable. Came to know about them from our known and now same ones will help in our family business campaigning also.