A/B Test Case Study: iOS App Purchase Screen
How a simple design change led to more than 10% increase in conversion
In March 2018, I joined Simply as a product designer. Simply is a music learning service that helps people learn how to play musical instruments in a simple, fun way.
At Simply, we work in squads, inspired by the Spotify Squad Framework. We call them ‘pods’. each pod has a different KPI in mind. I was assigned to the Conversion pod, where the KPI was, as you can guess, in-app conversion rate.
As a designer who mostly cares about UX + delighting the users, I wasn’t very happy at first. I thought of conversion as something much more related to marketing than thoughtful product design that solves actual problems. I strive to make the world a better place using thoughtful, empathic design, and not to help a company sell more.
All of these thoughts changed in a few days. The main reason for the change was when I saw that our main app, Simply Piano, is rapidly growing and is very loved by its community. I understood that I’m working for a music+education tech company, which is basically all I could ever ask for.
Then I got my first task, which was to redesign the pricing page in the app. Once I started the research stage, I saw that the potential to make an impact could be just as exciting as any other part of product design. Thankfully, it ended up as a success. Here’s how we did it:
Step One: Learn from others
As usual, before I started working on my design, I wanted to learn more about the task. ‘Redesign the purchase screen’ is a very broad task, and I didn’t have a Product Manager to tell me how to do it.
One of my favorite ways to learn is by looking at other companies and seeing how they solved the same problem. I examined many examples of how other companies sell their subscriptions, and it all brought me some cool ideas and helped me make better decisions later.
Step Two: Understand potential flaws with the current solution
In order to test an ‘improved’ version, I had to first assume what the problems are with the current purchase screen. Sometimes the assumptions can be based on data, but it’s usually harder without conducting proper A/B testing.
My main assumptions were:
- The CTA is not dominant in the screens
- The white icons don’t convey a strong message. They’re hard to translate into meaningful insights.
- Reading a list horizontally is hard.
Step Three: Quickly ideate different concepts
My next step was to open Sketch and try to quickly bring the ideas I came up with to life. I didn’t put much effort in the design, I simply wanted to visualize ideas I wanted to focus on. I mostly focused on:
- Social Proof: Show testimonials of subscribers who love us, using a video or just quotes. I found this as a common pattern mostly in marketing websites.
- Reducing friction: Make subscription one click away. I saw how many apps show both the offering and the plans in the same screen, so I assumed it could work for us as well.
After seeing previous A/B tests and doing some reading, I learned that some friction is usually not a big problem if it brings value to the users and doesn’t take too much of their time. The purchase screen is a task that usually happens only once, and people are not rushed when they buy a subscription — they read and think before making such a decision.
Step Four: Choose one solution to test
In order to save time and efforts for other tasks, we decided to leave most of the ideas for future iterations, and for now only focus on one solution which we believe in the most.
The final purchase screen consisted of two steps, one is a carousel which explains the benefits of a premium membership in a more visual way, and one lets the user choose a plan. The glowing colors over the dark background reminded people of Sci-Fi movies, so it got the original name ‘The Sci-Fi Version’.
Of course, I didn’t do it alone. My co-workers did an amazing job and greatly contributed to this test. While I did the design, I got help with choosing the right segments and KPIs, writing great content for this screen and having it translated into multiple languages, and developing it so it would work in all iOS devices. We started the A/B test of original vs. new purchase screen among 100% of our new users (50/50).
Results
We were closely following the numbers since day one, but we didn’t make any assumptions until the numbers were large enough. Eventually, we saw a meaningful ~10 % increase of purchases in the version with the new purchase screens. It’s quite a big number for a visual change alone, and was above our expectations.
It was also a great experience for me to work as my own boss and help make a big impact towards the growth of Simply. I felt that from day 1, the company trusted me and my skills, and it made me want to do my best.
What’s next
We knew we could iterate this screen and probably increase the numbers even more, but we were trying to make the biggest impact possible with minimal efforts, so we were skeptical regarding our chances there. We started coming up with other ideas to improve conversion. My second task in the team was even simpler to implement and made a bigger impact on conversion. More on that, in one of my next stories.
Yep, we’re hiring
Simply is expanding and we’re looking for awesome people to join our team. Our offices are in Tel Aviv, Israel. See open positions.
