Test strategy for free trials and introductory prices in Google Play
This is a guide to understanding introductory prices and free trials, along with their expected behavior. It is also a cautionary tale, inspired by a series of monetization experiments that I assisted in launching this year, as well as hours spent reading Android and Google documentation. I’ve seen plenty of good articles on in-app subscription testing in Google Play and none of them has yet covered special offers— a highly sensitive piece of functionality, as it turns out.
We’ll start with terminology, then I’ll explain the expected behavior, recommend some test data, and wrap it all up with an ultimate checklist for special offers. Along the way, I’ll point out the scenarios where bugs are most likely to occur.
Let me know if you’d like an article on the same topic but about the App Store!
First, let’s talk about subscriptions like a developer
Subscription — a certain set of benefits that users can access during a stated period. Users gain access (or entitlement) to a subscription by purchasing a base plan or offer. A subscription could have multiple base plans.
Base plan — a set of billing periods (monthly or annual) and renewal types (auto-renewing, non-renewing) for a certain subscription. A base plane could have multiple unrelated special offers.
Special offer — provides users with free trials and/or introductory pricing. While any user can purchase a base plan, offers are only available to users who meet the eligibility criteria defined by a developer.
Product — is a unique combination of subscription, base plan, and special offer (if any). That’s what a user purchases. Then, a purchase is followed by either an activation of a base subscription plan or an activation of a special offer and then a base plan.
What is eligibility and why it’s important?
Eligibility criteria answer the question of whether one could use a special offer or not. For example, if we only want to give a discount to users who have never subscribed before, we’ve just described the eligibility criterion. There are 3 use cases of eligibility: new customer acquisition, upgrade, and developer-determined. So far I’ve only had experience with a new customer acquisition: “never had this subscription” and “never had any subscription”. The article won’t cover other cases, but I hope you’ll get the idea and explore them on your own if necessary.
Make sure to ask the business about the expected eligibility criteria early in development
The product team may not be aware that eligibility criteria can vary or that eligibility even exists as a concept. If not communicated clearly, one of the worst-case scenarios could be showing a paywall with a special offer to non-eligible users and not showing it to eligible users, making the whole feature worthless. So, it is better to handle eligibility inside of a business logic.
Test cases like “user x is eligible for y” have the highest priority, while test cases like “user x is not eligible for y” have the lowest priority.
Users who receive a special offer by mistake are happy and likely to stay with the product. On the other hand, users who were supposed to receive the offer but didn’t, especially in the case of unexpected charges, are angry and tend to leave, slamming the door shut.
Expected behavior
So, we expect a special offer to align with the business goals and motivate our users to subscribe. We also should expect it to align with the Subscription Policy of Google, as the policy helps to stay direct and honest with users and should not be violated. Given the eligibility criteria defined by the business, I’ll give some examples of what’s a violation or bug and what’s not.
Note that there’s a known inconvenience (GitHub issue, StackOverflow 1, Stackoverflow 2) with the special offers testing. The same offer can’t be used twice with the same Google account, cause offer eligibility can’t be reset. My workaround for this is to separate purchase testing from offer testing. In my experience, for special offer testing, you don’t need to confirm a purchase. If something’s wrong with a free trial or an intro price, it’s usually seen on a Google Play native purchase widget before the confirmation. I also find switching environments very helpful. Let’s see what that means.
Environments
Each environment is a set of conditions for a unique behavior of an in-app purchase flow.
Test environment — The environment for internal testers allows developers to test the subscription flow for free using “mock” payment methods and shortened billing periods. To enable this, developers follow the instructions from Google, distribute .apk test builds through Google Play internal app sharing, and configure the licensed testers' list of Google accounts — a pretty standard procedure. Later I’ll explain why a test environment is not enough.
Production environment — The environment for actual users with real-world data, who download the public release build directly in the Google Play Store.
Production-like environment — The environment has real-world data but in a not-yet-released internal test build. “Mock” cards and renewal periods won’t be available, and sometimes that’s what we need.
Scroll down to the end to learn how to switch between test and production-like environments*
Introductory price
— a special offer that gives a discounted price for a set duration. Introductory pricing phases must be the same or lower than the base plan price. They can be absolute or relative to the base plan price.
Below there’s a great example of a paywall that aligns with the Subscription Policy. It clearly describes the terms of the offer, including the duration, pricing, and description of accessible content or services.
When we hit the “Subscribe” button we might enter 3 different scenarios.
- Google account is eligible for an introductory price offer
In the screenshots below is what we usually wanna see the most: a “Today” block with a discounted price, correct dates, and durations. Given the criterion, it is the expected behavior for a user who fits in it.
If I earned a dollar every time someone panicked and ran to me, claiming that we were billing people every 30 minutes, I could have treated myself to a nice designer bag.
Anyway, it is important to test with production-like time limits before deploying to real users due to ambiguous periods in the test environment. For example, both 1 month and 1 week are represented as 5 minutes in test.
2. Google account is not eligible for an introductory price offer
It is the expected behavior if the expected eligibility criterion is “Never had any subscription” and the user had subscribed before to a different subscription.
It is also the expected behavior if the eligibility criterion is “Never had this subscription” and the user had this subscription before.
However, it is a bug, if the expected eligibility criterion is “Never had this subscription” and the user had a different subscription before. In that case, a developer probably just forgot to set the eligibility and it stayed “Never had any subscription” by default. It wouldn’t be a violation of policy, just a mismatch with the business requirements.
3. The introductory price is not configured at all
In that case, the information on the paywall would be inaccurate and misleading. Therefore, it would be a violation. Note that the only differences in a production-like environment would be the payment method and duration of the subscription.
Free trial
- a special offer for a specific (monthly, yearly, lifetime) subscription that provides a designated number of days, weeks, or months at no cost. Free trial phases can range from 3 days to 3 years.
Another great example of a paywall is below. Here we explicitly let the users know how and when a free trial will convert to a paid subscription, how much the paid subscription will cost, and that users can cancel if they do not want to convert to a paid subscription.
Here we also may encounter one of the three scenarios:
- Google account is eligible for a free trial offer
Note that regardless of the actual free trial duration (a week, a month, a year), it will always be 3 minutes in test. That’s why it’s important to check it in a production-like environment.
2. Google account is not eligible for a free trial (or is not configured at all)
Here non-eligible users are treated differently. On version 6.0.1 of Google Billing Library, if I’m not eligible for a free trial I don’t see anything like the “You’re not eligible for a free trial” text. The user experience is the same as if the trial didn't exist at all.
However, I came across some screenshots from 2021-2022 (source 1, source 2, source 3), where "You’re not eligible for a free trial" was reproduced. In May 2022 a major update of both subscriptions and special offers structure was released in the 5.0 version of the Google Play Billing library. I am assuming this is related to the changes in trial non-eligibility handling. If anything, prove me wrong on this StackOverflow thread.
Recommended test data
Keep a test Google account, that never subscribes to test eligibility criteria “Never had any subscription”
Let’s call it “basic” (basic@gmail.com), it should be in the internal testers group.
Once the purchase has been made, a test account won’t be eligible for a free trial again, and tricks like reinstallation or clearing the cache won’t change it. Also, there is a limit on the number of accounts per phone number, and adding accounts to an internal licensed group is a highly time-consuming process. So, it’s better to test purchases separately and keep a clean account to test this type of eligibility for every offer that comes.
Keep a separate Google account to test the initial purchase and eligibility criteria “Never had this subscription”
Let’s call it “pro” (pro@gmail.com), also in the internal testers group.
If your team is often experimenting with monetization, the best practice would be to have multiple accounts like this. However, there will be only one opportunity to use an offer for a specific subscription per account. You’ll have to switch to another account for a retest. c
Keep a non-licensed account to test the prod-like duration of new subscriptions and special offers
Let’s call it “default” (default@gmail.com), not in the internal testers group.
Considering that test durations are ambiguous, it is better to ensure we are not launching a weekly subscription instead of a monthly one or 1 month for free instead of 1 week.
Let’s go wrap it all up in a checklist. “pro”, “basic” and “default” are coming with us.
Free trials and intro price offer’s checklist
New offer — offer settings, error handling, and initial purchase
Test environment:
- Free trial or introductory price eligibility. There should be another expired subscription on “pro” account. If it’s “Never had any subscription,” make sure “basic” sees it and “pro” does not. If it’s “Never had this subscription,” make sure both “basic” and “pro” see it
- Intro price. Is it an absolute amount, such as 5$ off the base price, or a percentage discount?
- Base pricing. You could also check how prices are being converted to different currencies to make sure they aren’t hard-coded
- Error handling. Purchase with a test card that always declines
- Successful purchase. Purchase with a valid test card on the “pro” account
- Renewal type. If it is not renewing, we should not see any renewals once the test subscription duration has expired. Otherwise, there should be 6 renewals. In a test environment, a subscription by design only renews 6 times and that’s okay.
Prod-like environment:
Scroll down to the end to learn how to switch to prod-like*
- Free trial period.
- Intro price period.
- Billing period.
- Successful purchase. Buying your app subscription in production is a very common and valid testing practice. It is only recommended when subscriptions are being launched for the first time
Existing offers (regression)
In my experience, it is sufficient to do basic integration tests for regression. There is no need to test all offer settings, to test in production, or to test every single offer.
Test environment:
- Error handling. Purchase with a test card that always declines
- Successful purchase. Purchase with a valid test card on the “pro” account
* How to switch to a prod-like environment in an internal test build
It’s not possible to access the internal test build if you’re not in an internal testers group, and there are shortened billing periods for internal testers. So, normally you won’t see real-world billing periods in an internal test build. But there’s a workaround.
You’re going to need 2 Google accounts:
- an account in the internal testers group: internal@gmail.com. Let’s call this one “internal”.
- and an account not in the internal testers group aka a default personal account: default@gmail.com (a real payment method should be linked to this one, we’re not gonna use it though). Let’s call this one “default”
Then
- Add an “internal” account in Google Play Settings.
- Download the test build (I am assuming developers share test .apk builds with internal app sharing).
- Add a “default” as a second account in Google Play Settings.
- Remove “internal” in Google Play Settings. This step is crucial because, no matter which accounts you switch to, Google Play will prioritize internal@gmail.com over default@gmail.com and put you in a test environment. If you have a look at the official documentation, you’ll find the following statement from Google: “If the device has more than one account, the purchase is made with the account that downloaded the app.”
- Open your app under test, go to a subscription entry point, and click the “subscribe” button to invoke a Google Play purchase widget.
- Boom, you’re in “production-like”! Free subscriptions are disabled.
To return to the “test” environment with free subscriptions, you should remove “default”. Then, add “internal”, and restart the app under test.
