Apptopia Data Algorithm (Part 2)

Last week I started to detail the inner workings of our estimation algorithms. This post will continue to dive deeper into the key variables we use in our models which have the most predictive power.

The goal of this series of posts is to help you get comfortable with how our business has been able to achieve such accurate usage estimates without building a mobile panel or collecting user / device level data.

App Functions vs App Categories

Since the inception of the App Store & Google Play, Apple and Google have created a category driven taxonomy to bucket all of the different apps in.

This taxonomy is clearly something which neither app store views as very important because over the 10+ years the app stores have been in existence, I can count on my hands how many meaningful changes have been made to the category breakdowns. The problem here is that the taxonomy they built is so generic that it’s nearly useless. For instance, what is an “Entertainment app”… it could be TikTok, Netflix, XBOX, Fortune Telling App, ScreenMirroring, or even a meme creator app. Given such an extremely wide range of apps, how exactly does the “Entertainment” bucket help a user?

Lets dive deeper into the Travel Category and start to talk about how this taxonomy issue created a market opportunity for us with our models. If you were to look at some of the top Travel apps today, you’d likely see these players consistently at the top:

— Uber (#1 rank)

— Yelp (#3 rank)

— Expedia (#5 rank)

Today Apple & Google view these apps essentially the same — “Top Travel Apps”. We believe that our competition views them similarly, and so when they are producing models they are doing so at the App / Store / Country / Category level. This means that they are likely building a Pareto Curve for the Travel category which states that the #1 ranked app gets X and the #2 ranked app gets Y. They all follow a certain curve with reduced values as rank gets lower.

The issue with this is that Travel is just a store category and has no indication on the actual function of the app. When you think about the usage pattern for an app (how often you open it and for how long), it’s based less on the category of an app and based more on what the app actually does, or its function. For instance, you might open Uber quickly to call a ride and then be done with it. In contrast, you might spend 10+ minutes researching vacation options for an upcoming trip in the Expedia app.

At Apptopia, one of the ways we’ve achieved such accurate usage estimates is by taking a different approach. We started by building out a proprietary Natural Language Processing (NLP) Algorithm which clustered and re-categorized the entire app store into “Function Specific Categories”. We use the App Name, App Description, and “Whats New” text from version releases to extract the keywords which are most representative of what the app actually does. The result is that when we look at the same “travel apps” above, we now see:

— #1 Ranked “Ride Sharing App” (Uber)

— #1 Ranked “Restaurant Review App” (Yelp)

— #1 Ranked “Travel Aggregator” (Expedia)

We built and trained our Usage Models (DAU, MAU, Sessions, and Time Spent) on this new proprietary FUNCTION level versus the generic category level. This allows our models to actually understand what the app does, and use more closely related apps as benchmarks, while producing our usage estimates.

User “Reactions” Are Key

We fundamentally believe that Ratings & Reviews are one of the least utilized (and most signal rich) data sets today. A rating / review (regardless of shape, size, color, sentiment, etc.) is simply a user reaction. Typically the more people who react to something, the more they care about it (hence why they say the line is so thin between love and hate).

So over the years, one rather rich variable we’ve used to estimate usage is “Downloads / Ratings” or rather how many new downloads it takes to get one user reaction. Please note that:

— To date we have collected data on over 400 million unique user reviews.

— We have built technology which allows us to identify (and remove) “bot reviews”, so you should feel confident the review data fed into our models is based on real user actions.

For each function we create a benchmark of new users it takes to get a new reaction, and then we compare each app in that function to the benchmark. For instance, let’s say we were looking at the Puzzle Category and we saw that on average it took 300 new downloads to get 1 user reaction. If we saw Puzzle App A starting to get new user reactions (ratings / reviews) at a rate of 1 every 200 new downloads, we’d be able to deduce that users are likely spending more time inside that app vs similar apps.

One of the more interesting developments in this space is Incentivized Reviews. If the user gives an app 5 stars, the app will give the user some in-app currency or something else in return. The result is that reviews actually hold even more predictive power now as it means that someone cares enough about the currency inside an app to take an action they ordinarily wouldn’t have taken. This also means that as a result of getting this currency, they are likely to use that and spend more time in the app as a result. Imagine if the Princeton Review app gave one free test prep course to anyone who gave the app 5 stars. You would be able to clearly state that more reviews = more users taking practice tests = more engaged users spending more time in the app. Hence the movement of this ratio over time has proven to be a very strong signal.

Why Risk Sharing These Secrets With the World?

(i.e. our competitors)

This is a question I received quite a few times over the last few weeks, as folks internally and externally were concerned about the risk of sharing our trade secrets.

The answer is a simple one; knowledge and execution are two wildly different things. My business partner Eli and I have been building this business for over 8 years together. Among the many things we learned is that our team’s industry expertise was a rather large moat (barrier to entry). Even if we gave you all of the data we’ve collected and the insights from these posts, it would still take you years to produce models which are as accurate as we have today.

It’s the same reason why my Bolognese sauce doesn’t taste remotely as good as Bobby Flay’s, even though he told me exactly what to do and gave me all the ingredients.

We’ve bled a lot to get to the point where we are today, and we do not believe others can easily replicate the success we’ve had. As such, our #1 priority is to make sure our customers and prospects have a deep understanding of how our special sauce is made. As consumers of data and software ourselves, we always choose to work with and partner with companies that are the most open and communicative with us.

We are data geeks at heart and we promise to keep this as part of Our DNA, and never take any shortcuts which put our business, your business, or the sustainability of our data in any danger.

As always, we are here to help if you have questions.

Cheers,

JK