Increasing user onboarding by 233%
BABYWORLD OVERVIEW
Babyworld was a pet nurturing social game targeted towards the female audience. With 3.5 million users, 350k MAU and 500k facebook fans, it was the largest social media game from a south asian studio in 2013, created by a tiny team of 5 and grown with $0 advertising spend.
GOAL OF ONBOARDING
- Get at least 60% users on-boarding compared to 30% of the initial tests.
- Increase D1 retention by 30%
- Help users engage with the game, discover all parts of the game world.
MY ROLE
- User Experience Designer
- Onboarding Analyst
- Onboarding / Game Programmer
OVERVIEW
To accomplish this we went through a series of experiments until we found the right on-boarding for us. Each experiment had multiple stages.
- User personas & user testing (in some)
- Analysis
- Experiment
- Results stage
Following was the process of creating our ideal on-boarding:
- Zero tutorial.
- Open world guided tutorial.
- Closed guided tutorial.
- Closed guided tutorial with quest system.
FINAL RESULTS
- 233% increase in on-boarding completion (Initial 30% — Now 70%)
- D1 retention increased to 30% percent from 15%
- Users engaged with the questing system leading to discovery.
THE PROCESS
01 ZERO TUTORIAL
No tutorial was setup to create a baseline for how much content would the users consume and their D1 retention.
EXPERIMENT
Babyworld was flash based, since no off the shelf analytics (funnel) tools were available, so we built one ourselves.
An A/B test was set up. Users entered either into the game or the avatar generator. To get a measurement of which worked best {our bet was on the avatar generator, since they already invested time in making a baby and interacting with it they will stick longer}
EXPERIMENT RESULTS
Experiment ran for 7 days with the following results.
- D1 retention baseline at 20%
- User Onboarding baseline at 30%
- Users sent to avatar generator had higher D1 retention.
- No numbers on basic game play completion.
02 OPEN WORLD GUIDED TUTORIAL
Ideally we wanted our game design to work without a tutorial and we wanted users to figure it out. That clearly didn’t work and does not work with social game.
EXPERIMENT
The next step was to create an open world guided tutorial, where the user was shown a pointer to do certain activities.
- The user can chose not to follow the steps. After a certain time the tutorial would shut down automatically.
- The user wasn’t directed or guided for everything, e.g the user isn’t told to customize the avatar step by step but he can choose to change one thing or a dozen.
- The guide will be enabled whenever the user does an activity for the first time inside the game.
- User events recorded with granular detail and including the preloader to find out drop off points in more detail.
EXPERIMENT RESULTS / ANALYSIS
Bottomline nothing changed. No improvement in retention/engagement. But the new numbers and events gave us some valuable information:
- Largest drop off at preloader: The game never loaded for users on Firefox. So people just waited and dropped off. [fixed the bug]
- Second largest drop off at avatar creator: Many users never put a single item on the baby. They didn’t understand they needed to drag not click [we allowed both in the next experiment]
- Our audience wasn’t older females: Baby world was created for the older female audience who played farmville a lot. But we attracted a large young female audience. This had us look at our game and marketing in a different light. [one reason was our art direction kids loved the art]
- No marked difference in retention/engagement of two groups: The two groups one who followed the tutorial and the ones who didn’t follow the tutorial didn’t know show any large difference.
- More tutorial followers made it to the game: Users who did complete the avatar mostly didn’t drop off after it and made it to the game.
03 GUIDED TUTORIAL CLOSED
From our findings in the open tutorial we did the following changes:
- Remove open tutorial, replace with closed tutorial where users have to follow the steps to get to the room and they cannot deviate from it.
- Guide the users not only with pointers but text instructions.
- Keep it interactive introduce a mascot which guides them along the way.
USER PERSONA’S & TESTING
From the data of the experiments. We created two user personas:
Middle aged not too tech savvy lady
- She’s got kids of her own.
- She’s patient.
- She reads and follows instructions.
- She likes cute babies and kittens.
- She plays games like farmville and other social games that let you choose your pace.
- She’s not competitive but likes to talk to friends and enjoys neighbors.
Young girls tech savvy
- Not patient, if something is blocked she will leave.
- She likes cute babies as dolls.
- She plays children’s games with low depth.
- She’s quick to jump out if she doesn’t like it.
- But she would stick if her friends are here.
EXPERIMENT
We tweaked our designs based on our profiles and then we went to a large company with a number of middle aged women. The one difference was these were tech savvy women.
- We tested our game on nearly 200 users.
- Each test conducted individually in two sets of environments one of us sitting with them and other with a screen recorder with no one around.
- After the experiment everyone was given a form to fill asking them specific questions about the tutorial sections and multiple choice feedback.
EXPERIMENT RESULTS / ANALYSIS
The user testing was the most useful thing we did. We found loopholes in our implementation and importantly the understanding of the tutorial. Following were the findings:
- Avatar creator was still the largest users drop. The reason? Lack of consistency in the tutorial. Users were made to follow instructions click by click. Then allowed roaming without an idea of how to get to the next step. Solution: Fewer choices at the start, they can always come back and make a better avatar.
- Dragging tool tip isn’t intuitive. Half of the game play is based on dragging. Solution: Improve the visuals of drag and aid it with text.
- Users enjoy the mascot. Solution: All instructions should be through the mascot.
- The little gamification is working. Solution: Integrate the quest system with the tutorial gamify the whole tutorial.
- User enjoys the visual feedback in terms of currency etc but its less. Solution: Add small but more visual rewards at each correct step in the tutorial.
04 GUIDED TUTORIAL WITH QUEST SYSTEM
After reviewing the recorded sessions, on spot feedback and the written feedback from the users.
- We fixed issues seen from last experiment.
- We integrated our quest system in the tutorial so user move on to discover more.
EXPERIMENT
We tested our tweaks on a new set of users from the same company.
- The users didn’t drop off at simple tasks (e.g dragging)
- Most users were able to finish the avatar creator (the second biggest drop off)
- Some new bugs were identified in the tutorial.
After fixing the bugs the new tutorial was shipped and the experiment ran for another 7 days.
EXPERIMENT RESULTS / ANALYSIS
- With the new tutorial system in place 70% percent of our users completed the tutorial which is 233% from our initial tutorial completion.
- D1 retention increased from 50% taking it to 30% which is still 5% lower than industry average.
Later we figured we can increase D1 Retention a lot more by giving user positive interaction with other human players and bots. Our next focus was on increasing retention by creating communities that worked both in game and out of game. You can read more on this here.
LEARNING & INSIGHTS
- Intuitive for you does not mean intuitive for everyone.
- Test as soon as possible.
- Be consistent with guidance, don’t increase or decrease within a tutorial.
- Give plenty of positive feedback in the start.
- Measure your perfect player what does he do? How do you make everyone do it.
- Have users invest time in creating the avatar at first. This time investment converts.
- Don’t leave users hanging after a tutorial funnel them in a quest or guidance system.
- Don’t let bugs and cross browser issues let users drop.
