How to measure your product’s in-app subscription performance

You can’t improve what you can’t measure. But what to measure is a question that daunts all product owners. You might be rejoicing looking at vanity metrics and thereby overlooking problems with your product. Vanity metrics are feel good numbers that really don’t give you any guidance. They are non-actionable and they just look good on paper. Mobile apps boasting about millions of downloads on playstore are a typical example of that.

If you are an app developer chances are very high that your app has in-app purchase option. There are four types of in-app purchases: Consumable, Non-consumable, Non-renewing subscription and Auto-renewing subscription. Apple has put up detailed description about these here. Gaming apps use consumable and non-consumable subscription options. But for rest of the apps, the most commonly used one model is the auto-renewing subscription option.

For the purpose of the article let’s assume the product has in-app auto-renewing subscription. The product owner has introduced a new trial to payment flow and wants to see whether this new flow has actually worked or not.

A brute approach to this would be to look at subscription/revenue numbers of a bygone month and compare them with similar numbers of a recent month. If you are not doing something woefully wrong with your product you would see your subscription numbers growing each month and you would feel all is well.

This is a typical case of falling prey to vanity metrics. Looking at absolute numbers will not give you anything. What you should be looking at is the subscription trend. Subscription numbers are highly correlated with traffic on your app. So, a better metric to look at would the ratio of subscription numbers and monthly active users : (# of Subscribers)/MAU or (Subscription Revenue)/MAU.

The ratio approach is good so far but the ratio itself is not quite right. The number of subscribers or Subscription Revenue also includes payment done by auto-subscribed users. So the increase in numerator is not a function of product improvement. We should be looking at subscription numbers as a result of product change. A better number to look at would be number of new paid subscribers or new user subscription revenue. Now if we draw a plot between # of new paid subscriber and MAU or New subscription revenue and MAU, we would get how our product is doing from subscription standpoint.

Let me know your thoughts in the comments below.