Apptopia & Mobile Data Have Evolved
Over the last two years, Apptopia has been uncharacteristically quiet. It has been a weird two years if I am being honest. Many companies struggled with their post-COVID identity and had to deal with squeezing margins and layoffs amid many excruciating challenges. Like others, Apptopia’s last two years have been wholly defined by War and Patience. While the tech sector and valuations crumbled around us, we simply followed our DNA….
“During times of uncertainty invest heavily into Data & Intellectual Property”
Our investment decisions led to us rearchitecting our data ecosystem and enabling the development of new, digital intelligence, data products that have a .94 correlation to the most impactful company KPIs.
Although our journey has been costly, it has more importantly been enlightening and transformative, laying the foundation for the future of alternative mobile data. Below are some of my key learnings and results from the last two years:
Today, you read more about “Data Privacy” than at any other time during my decade-plus running Apptopia. Between the rumors of TikTok stealing your personal data and every other state in America creating its own version of GDPR, it’s easy to feel like things are spiraling out of control. However, as someone whose legacy and financial livelihood depend on ethical data collection, I can tell you that it’s actually quite the opposite. Over the last two to three years I’ve seen mobile data collection get way less passive (i.e. sneaky) and be much more of a personal decision.
Panels pre-2020 were mostly VPNs & AdBlockers, most of which have since been taken off the store, shut down, or rebuilt completely. Today, we have access to so much transparency about personal data collection and our mobile footprint. There are many companies and marketplaces that provide monthly incentives of various value based on how much data an individual is willing to share.
I found this shift to be rather motivating, as it paved the path for higher-quality mobile panels (usage data from real app users). A higher quality, more stable, panel allows a business like Apptopia to really fine-tune our understanding of consumer usage patterns and identify key trends early on. Over the last 2 years, we tested over 20 different data sets and panel sources. Along the way, I formulated guiding principles and defined my areas of focus for identifying a strong panel:
– Diversity of incentive or functionality offered to end-users matters, as it results in panelists with different interests, demographics, and behavior patterns.
– State and country-level location diversity is important and unfortunately what is often reported by panel providers is not fully reliable.
– Analyze, question, and understand all extreme volatility in panel size or overall panel movements.
– Poor panelist retention is a deal-breaker, period.
One of our biggest realizations was that no one panel could check all of these boxes. We learned that if you want to use device-level data as a reliable signal to consumer behavior you need to build a master panel made up of many smaller panels. This approach is the only way to truly reduce bias and build a representative audience.
Warning: Along our journey, we met a lot of professional smoke and mirrors. Be skeptical until you have all the information. For instance, we evaluated a bunch of panels that offer historical data going back to 2018. Through testing, we found that nearly all of the data that was obtained on or before 2020:
a) Contained a LOT of bot activity
b) Have unacceptably high panelist churn
Both make it very hard, if not impossible, to have a stable panel of users which you can extract real signals from.
As you can see, we’ve built real muscle working with mobile device panels over the last few years. Most notably, our ability to extract a high-frequency signal from a number of different data sets and then unify them together to better understand the digital health of today’s largest companies.
Put simply, device-level data allows us to implement a dynamic and more granular usage signal into our core estimation models. Our customer base is hyper-focused on understanding the health of large businesses with established customer bases (i.e. Uber, Spotify, Airbnb, Paypal, Tinder, etc.) where downloads or new users are less often a leading indicator of company performance. Usage and understanding where key segments of consumers spend their time is the whole game, and this new iteration of Apptopia is hyper-focused on that.
We’ve developed a very powerful way to blend our proven approach of reverse engineering Apple & Google’s rank ecosystems with direct observations from our mobile panel (4m devices). The results have yielded a .94 correlation to public company KPIs, which was driven by some of the following major improvements:
1) Direct Measurement > Estimation
For key inputs to our model, we are now feeding in core measurements of user engagement that we directly observe from our mobile panel (previously these were estimated). Examples include:
– Retention
– Session Frequency
– Session Length
– Probability of Use
– Long Term User Behavior
A good example to take a look at is Social Media. You can now see extremely accurate views of actual minutes spent per day in an app as well as stronger signals of growth vs. peers.
2) Probability of Use Is Key
Any usage event can have a number of different origins:
– Existing user
– First-time user
– Dormant user
– Re-download user
In this release of our model, we materially deepen our understanding of these dimensions. Below is a great example of how this has helped further refine our signal. Look at the difference between DraftKings app performance during the 2023 SuperBowl with our old and new models:
3) An Improved Understanding of the Macro Environment
We’ve added a dynamic feature to our model to account for major economic events going forward. Two key areas of focus here were:
1) Improved understanding of changes or shifts that impact an entire function of apps (i.e. Travel during Covid).
2) Improved understanding of mobile penetration per country and how this is evolving over time (i.e. Month by month).
Below are two really good examples of these improvements:
It’s All About The People
It feels sacrilegious to talk about the importance of People, in a world where even mentioning the word “AI” likely results in a term sheet. However, one of the evolutions at Apptopia that I am most proud of is the collaboration we’ve stood up between Product, Engineering, Data Science, and Research. Over the years I’ve seen the power that every single one of these functions can contribute to our intellectual property. However, at the intersection of all four, there is a uniquely healthy friction. And healthy friction done right creates something uniquely special. We believe this team is one that can have a continuous impact on the quality and accuracy of our estimation algorithms for a long time to come.
What should you expect from us going forward?
We will continue to evaluate new data sets, test new ideas, and continue to improve our core signal on consumer behavior. Moving forward you should expect our models to be dynamic and capable of adjusting to the ever-changing world we live in.
For those of you who know me, or have worked with me; you know I’m kind of a data geek. I love this stuff. So please don’t hesitate to reach out if you have questions, want to share stories, or simply want to hear how passionate I am about the future of Apptopia.
JK
