4 Reasons why mobile ad attribution numbers never match up



4 reasons why mobile ad attribution numbers never match up - the mobile spoon


[This post has been modified on September 2019 and now includes a 5th reason]


So your amazing mobile app is out there, beautifully designed, well developed, except nobody knows about it.

Rest assured the zombies are not coming, so you need to bring them through different marketing channels: ASO, social networks, user acquisition, viral campaigns, events, whatever works well for your product.

In order to know what channel works well, you need data, but the data you need doesn't reside in one place.

Facebook, Instagram, Twitter, Google, they all have their own dashboards with different measurement policies, while you, as the owner of one app, struggle to understand your overall return on ad spend, searching for a reliable, accurate and consistent source of attribution numbers.

Our approach: having a single point of truth  

For us at Missbeez, the strategic decision was to bring back the attribution data from the different marketing channels we use to our own system of records.
We have implemented it by using AppsFlyer, a leading SaaS mobile marketing analytics, and attribution platform.

Doing so allows us to:
  1. Reduce our dependencies on multiple external tools and dashboards 
  2. Track installs from all marketing channels (paid/organic, digital/non-digital) in our own consolidated dashboard 
  3. Have the attribution data embedded within our core data model, ready to be used by our core business logic.
  4. Analyze the behavior of our marketing channels in the long run (weeks, months, years), in terms of retention rates, frequency of use, virality effect, and other metrics that are at the core of our business models. 

Running into discrepancies   

This process revealed what many mobile marketers already know: the numbers do not match. As written in this blog post by Adjust:

It’s an issue that arises on every single analytics platform - in one dashboard you have 5005 installs, on another, 7246. What’s the deal?

After doing some digging, talking to experts and exploring our Facebook ads, I would like to try and explain (also as a self-persuasion), why install numbers as shown by Facebook (and others) will probably never match the ones companies see in their own systems.


So there it is:

4 Reasons why mobile ad attribution numbers never match up


Reason #1: Facebook counts installations while you count users

If like us, your system keeps track of new users then I guess each user needs to do something before he becomes one.
Whether it’s adding an email, entering a nickname, or even just skipping the welcome tour - users might drop along the process, causing discrepancies between your data and other systems:
Some systems count the number of installations regardless of opening the app (Apple and Google). According to AppsFlyer, 10% of apps are installed but never opened.

Some systems count installations only once the app launches, regardless of what the user actually does when opening the app (i.e. Facebook).
Some systems (like our own) only count a user as an actual user once he completes a short verification step.

So continuing with our own system as an example, in the case of low engaged users that didn’t convert to fully active ones - the numbers might be completely different in each system:
  • Users that installed the app but never opened it will still be counted by Apple and Google as an install (but not Facebook and our own system).
  • Users that opened the app but didn’t go through the verification step will still be counted by Facebook (but not our own system)
  • Only users that opened the app and finished the verification step would potentially be numbered correctly by all systems. 

Now that we understand that, what differences should we expect to see? 

It depends on the drop rate during the validation/registration step, which depends on the level of complexity of this step.
Accurate benchmarks are hard to find but I did find the following:
  • 54% install-to-register conversion rate in games (according to LiftOff)
  • 35%-47% install-to-engaged conversion rate (iOS, non-organic users, according to AppsFlyer )
Looking at our own numbers (using MixPanel) it seems like 85%-90% of our new users convert from install to register.
  • 5%-7% drop during the tutorial (indicating some users didn’t fully understand what the app was about prior to installation) 
  • 5%-7% drop during SMS activation process (which makes sense as some users may not feel comfortable with this validation method but we do need it at this point).  



Reason #2: Facebook shows inflated numbers because some installs are counted more than once 

Ignoring the low engaged users who don’t even finish the verification step - what else can Facebook count that we do not?
Well, since our system counts ‘users’ and Facebook counts install - there may be more weird scenarios resulting in Facebook showing inflated numbers:
  1. Users upgrading their devices (every 21 months) will create additional “install” events (counted by Facebook) without increasing the total “users” count.
  2. Users installing the app on multiple devices will create additional “install” events
  3. Users re-installing the app will trigger extra installation events without increasing the “users” count. 

Note: according to Tune - re-installs account for 40% of all installs.
  • 18% say they deleted an app because it was buggy, and they reinstalled to force a reset to a better-functioning app
  • 13% say they deleted an app that they later found out they needed … and therefore re-downloaded it.
  • 6% say they deleted an app by mistake and had to reinstall it.

It might be an exaggerated number (for marketing purposes), but even if we take 10%-20% re-installs it’s still a lot!

So to summarize: we have device upgrades, multiple devices, re-installs, all increasing the number of installs as reported by Facebook and the likes, without bringing a single user in. How unfortunate…



Reason #3: Facebook’s attribution time window is much longer than you would expect 

Facebook app installs are logged when one of the following occurs:
  • A person clicks on an ad or link and installs the app within 28 days
  • A person views an ad or link and installs the app within 24 hours
(source)
Since smartphones are basically a wild playground for interruptions, users may see your ad, and immediately get diverted by the next item on their feed or by any other incoming thread on their smartphone. Such users might later go back to the AppStore (few hours or few days later) and install your app.

Such scenarios will be counted by Facebook as installs but depending on your implementation (there are few ways to address this issue) they may not get attributed in your own system.
The date reported as the 'app install time’ can also be deceiving because (according to Facebook): “The date used for reporting app installs is either the day the ad was last clicked or viewed” which can happen up to 28 days before the app was actually opened.

If you are measuring your ad performance on a weekly basis - remember that time differences can cause a lot of discrepancies.



Reason #4: Technical glitches 

Passing back attribution data to your own system is a bit tricky. We do this using AppsFlyer, as AppsFlyer manages the integration with thousands of different ad networks.
From a technical standpoint, this is not a 100% bulletproof since the API’s are based on remote events (coming from AppsFlyer servers), and by the nature of mobile apps events might get interrupted by real-world scenarios (disconnections, app termination).

Without getting too technical, I will just say that despite all attempts to solve those challenges, I’m 100% sure there are rare scenarios where the attribution data will not be accepted, resulting in mismatches and inconsistent numbers.


[NEW]: Reason #5: Apple LAT

Apple's Limited Ad Tracking (LAT) allows users to opt-out of having an ID for advertisers. It's a feature Apple added to improve users' privacy and gained a lot of popularity in the past 2 years (following the likes of the Facebook–Cambridge Analytica data scandal, which raised public awareness on the importance of data protection and privacy) and is now being used by 15%-34% of iOS users (15% according to AppsFlyer, 34% in the US according to SearchAdsHQ).



Once users are using LAT, it's impossible to track their source, which means that if they came from Instagram or Facebook through paid ads, they are still counted as organic installs.

Now that those numbers can reach 20%-30% of the users - it's meaningful.


Conclusion

There are at least 5 reasons why your attribution numbers don't match up, and still, I believe it's a good practice to integrate back all the attribution numbers into one system of truth.

Keep in mind that the truth, in the case of mobile attribution, might not be 100% accurate, and instead of locking down on the actual numbers - focus on the trends.


More on mobile: 




And before you leave, if you haven't subscribed to my occasional newsletter yet - you don't know what you're missing!!! (Not to mention you are probably not 23% more awesome than average...)







Comments

Gil Bouhnick The Mobile Spoon
Canz Marketing said…
Amazing write-up! Thankyou for sharing such useful and informative thoughts of yours on this topic.
facebook ads