7 Lessons from a Product Manager about Mobile A/B tests

First — Why should you run A/B tests?

You should always test your assumptions at key interaction points. Not every small decision and icon must be tested, but at major transactional points, small assumptions could account for large amounts of users or revenue lost. For example, the text on a button on a purchase screen can change the conversion rate drastically. Onboarding flows, core user journeys to most wanted responses, and purchase opportunities should be key areas of focus to test.

This article will cover takeaways I have gathered from a product perspective. You may be asking yourself, how do I setup an A/B test? For a tutorial on how to set up A/B tests with tools like Firebase, the tool I have worked with the most, check out this great article.

Pick your words (or what you measure) wisely

If you are implementing a new flow in your app with the goal to achieve the most wanted response (Sharing a scan in a scanning app for example) this is an obvious KPI. However, you should make sure to always measure key events that capture retention (Firebase has its own built in that you can choose), signing up for purchases, removing the app, and in the case of subscription apps with a trial, returning after the trial. You may acquire more shares by…