Pocket Gems APM Exercise

Ethan Reznikoff
Nov 1 · 5 min read

Prompt: The following questions are geared towards assessing your ability to approach ambiguous problems in a structured way, and to make a decision based on data.

Please download the Excel file included in this file: https://drive.google.com/file/d/18bjCBhJWTLyjOqDg8Oz9C7BoE4199a_t/view?usp=sharing

  1. How is the game doing since launch?
  2. Based on how the game is doing, what are some recommendations you would make to improve the product and why?

My answer:

To assess how the game is doing since launch, I’ll need to define and analyze metrics for user acquisition, retention, and monetization.

The game launched November 1, 2017 and we have data through March 15, 2018. That’s approx. 4.5 months, or 18 weeks. Dataset contains 160,067 total sessions, 31,101 unique users.

Acquisition Metrics

Figure 1
  • 31,101 installs
  • Weak installation growth
  • Initially fell from Nov to Dec
  • Grew from Dec to Feb
  • Tanked in March (why..?)

Retention Metrics

Figure 2
  • 16,814 (54%) of users only played one session
  • The distribution of repeat user activity is better than a typical 80/20 split where the lightest users make up 60% of all repeat users. Of the remaining 40%, the heaviest users make up 22% (9% of repeat segment), indicating (again) better than typical activity distribution
  • Avg sessions per user (all): 5
  • Avg sessions per user (repeat segment): 10
  • Avg days to churn (all): 8.3
  • Avg days to churn (repeat segment): 18
  • 1/7/30 day retention = 46%/24%/9%
  • Avg monthly churn: 91%

In figure 3, we can see clearly the general increase in both DAU and MAU up to late-February. If we remove all one-time users from the calculations (not shown) we see less of an increase over the full 18 weeks, but an increase, nonetheless. In a practical setting (seems superfluous for this exercise–hope I’m not losing points here!), I’d compare data for each user type by frequency, and then segment further by device and country to see where we might be missing opportunities.

Figure 3

This is a basic indicator that the game has generally been doing well, though we may have reason to be concerned about the decrease in activity in late- Feb/March.

In terms of stickiness, we can look to DAU/MAU for an idea of how we’re doing with habit formation. I took the average DAU (including one-timers) for each month and divided by the MAU number to get 5 data points.

Figure 4

At first glance, we can see we’re in the 10–15% range, which I believe is okay but not amazing for mobile games. It’s a good sign that they’re increasing, though. I’ll want to dive into these stats for various user segments, too, to get a sense of the impact of potential correlates like country and device used.

Earlier, I referenced our 1/7/30 retention metrics. Figure 5 outlines our metrics compared to industry benchmarks. We’re exceeding 1 and 7, but just shy of 30.

Figure 5

Ultimately, our DAU/MAU, as well as our 1/7/30 day retention metrics tell a nice story about our stickiness. So far, acquisition metrics seem to need the most attention. Now onto monetization…

Monetization Metrics

  • Generally speaking, doing quite well…
  • 4.6% of the total user base has made at least one transaction
  • Average revenue per paid user is quite high at $37.22
  • Not surprisingly, I found a strong correlation between the 4.6% of users making purchases and the 4% of 30+ day users
  • The data seems to support my earlier hypothesis — we have bad acquisition metrics but for our most frequent users have very high conversions
  • And now we can confirm that much of our monetization is coming from that base of power users
  • Revenue growth likely affected by decrease in installations
Figure 6
Figure 7, Figure 8

Conclusion

Acquisition

  • Very weak installation metrics (especially recently) — March is almost 50% of November
  • Unlike retention curves, acquisition metrics should be increasing, or at least remain relatively steady — instead they’re trending downward, with March installations approx. half of November
  • We should devote most of our efforts here since the rest of our funnel is functioning well, but all falls mercy to the fate of installation metrics!

Retention

  • Retention metrics are very good — our 1- and 7-day retention curves exceed the industry standard, and our 30 day isn’t far behind.
  • DAU and MAU are growing steadily apart from the February drop, which seems to be due to decreased installations
  • DAU/MAU is very good for the mobile gaming industry, and indicates solid stickiness

Monetization

  • Most IAP revenue comes from “power users”, the 4% that played over 30x
  • We hit every industry benchmark here
  • The correlation between retention and monetization is an interesting on
  • I’d love to figure out if making purchases drives retention… cognitive dissonance tends to push us toward behaviors consistent with our actions
  • If this is the case, we’ll want to drive as many users into the monetization phase of the funnel as possible
  • Of course, focusing on customer acquisition will help this
  • But perhaps we could also lower IAP prices from $0.70 minimum to hook in more price elastic users
Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade