Increasing app engagement by 75%
OVERVIEW OF AGNITUS
Agnitus is an award winning educational startup / app for preschoolers with over 3M users and a place into top 5 on the apple store. The agnitus app built on common core standards is a complete educational system complete with 100 + educational games,books, assessment and planning dashboard for teachers/parents and feedback report cards. Funded by social endeavors of Eric schmitt & social capital of Chamath Palihapitiya
GOAL
- Increase app engagement by at least 50%
- Increase app retention.
- Find game completion rate and improve it.
- Improve app experience in general & each game in particular.
ROLE
- Game Producer
- Game Designer
- Product Analyst
OVERVIEW / TLDR
The first step was to identify the issues causing the low engagement rates than fixing issues and building on it. Here is an overview of our analysis and what we did to improve engagement:
- Analytics: We built our own analytics system & created reports on user behavior.
- Level design issues: Level design issues were identified, causing users to not progress as intended and fixed.
- The candy shop problem: A large choice made the children play with everything for a short time. We fixed that by introducing gamification in the system.
- Alphabet teaching common core issue: Alphabets weren’t taught in the traditional ABC format but based on phonemes. Secondly, multiple games worked in a loop to teach one alphabet in multiple ways. Parents & children didn’t understand that. They wanted more visual simple progression.
- Parents teaching behavior identification: Children had no motivation to do the work but do the activities which were more fun. Parents didn’t check on progress of the child. But when they wanted control. We gave more options to parents and more gamification for children to keep them motivated.
RESULTS
What worked:
- App engagement increased by 75% through gamification.
- 20% users started using the new level selectors.
- Game completion rates increased by 40% through level improvements.
- 30% decrease in bounce rate.
What didn’t work:
- Gamification didn’t cause a significant increase in retention.
- Parents turns out don’t like to be told grades.
THE PROCESS
Analytics Setup
- No analytics tool to analyze apps within apps: There are many analytical tools available to see analytics of a single app or games. But there are no solutions available in the market to get analytics of apps within an app. So we had to build one of our own to see data to improve each game.
- Cleansing data and generating reports for 150 games: An analytics tool was built in house to get game level analytics and build reports on that. After it data of each report and game was verified and cleansed to get the actual picture of the health of the application the first time around.
Level Design Problem
A major portion of our users fell after learning to say 1,2,3 and never got to 10. There were three parts to that happening:
- The leveling system in each game wasn’t visually presented. In theory, levels didn’t matter for preschoolers. They couldn’t understand the concept they just wanted to play. That wasn’t true, children were bored by the same activities they didn’t know the change or how much they had to play.
- The repetitiveness for teaching purposes (e.g at 8, you wouldn’t only learn 8 but also 2,3,5) made levels too long and with the short attention spans children quit the game. On re-entry levels were reset and back to 2,3,5 before 8 and reset.
- Play testing was done on children but all levels were tested by adults. There were level difficulties the children couldn’t pass and were stuck in a loop.
Level Corrections
- The repetitive levels were broken into smaller parts so even if the child quit the game. He started back where he left off instead of doing the whole thing again.
- The progressions were reduced and fault tolerance corrected for games to reduce difficulty.
- Level reports were generated for each game, levels with issues were analyzed, fixed and tested working with the educators.
Level Selectors, Visual Progress and Gamification Introduced
Leveling was introduced in every game. Children and their parents knew which alphabet or number they were on and they could go back and play the old letters as many times as they wanted through the level selector.
- Adults could also unlock next levels manually if they wanted.
- The negative feedback was toned down and turned into hints to help the children progress along the way.
This gave more control to parents and to children so they can play their favorite number, while knowing that there are still more numbers to come.
The Candy Shop Problem
User flow analysis showed preschoolers act like they are in a candy shop when given multiple game choices. Trying every candy, not finishing any until they are exhausted.
Children played every game but just a little, moving on immediately if they couldn’t figure out a puzzle in under 10 seconds.
This wasn’t bad if we were just a game. It was essential that the children progress in every game to teach them all the skills needed to learn the curriculum.
To fix the candy shop problem we introduced gamification in the app.
Gamification Rules
These were some gamification rules.
- In each educational category only limited number of games were open at the start.
- The fun activity games (coloring, playing with mascot, dress up etc) were set up as a reward. You get ‘A’ right and you can play dress up for a minute.
- Children earned stars for as a reward for doing good in a learning game. Enough stars and you get rewarded a fun game.
- A meter was introduced which told you how far away were you from earning a new game.
Gamification Theory
- To succeed it was essential that the children complete things. We did this by building a simple habit loop. Icon trigger, activity and reward in positive feedback, coins, progression and new games.
- A new game wasn’t a chore anymore but a reward you had to earn that new game and you had to earn that fun time activity.
- Playing the same numbers / alphabets in every game wasn’t a chore because it was a new game with its own novelty and puzzles.
- This took them to half the game, the rest of the game was completed in the anticipation of the new game or the quick reward of an activity game.
- This anticipation of a new game combined with visual and audio feedback for the unlock created a positive reward which the children craved for.
We A/B tested the app with the gamification and without.
The ABC song problem
Intelligent loop issue
The Agnitus app is an intelligent design to teach a user one thing in various ways. If the kid is learning about ‘2’ he will be taught 2 in 8 different games in a loop and by the time he is done he would know 2 perfectly.
Kids it turns out drop out. A lot. They either love one game and want to play it over and over again. Even if they are playing the game in the loop they might get bored at game number 5 (out of 6) and dropout. The system will start them again in the first game until they complete the loop.
Adults at time wanted to teach one game for say half and hour. The app kept pushing the children to the next game. This was the case for both numbers and alphabets.
The phonics problem
Traditionally alphabets are taught in a musical pattern, the order has no real learning significance rather a cultural one. Our educators created an alphabets teaching system based on motor skills & phonics. The easy letters come first then progressing onto the hard one’s. The method scientifically proven to enhance learning.
Schools loved the program, teachers understood it and trusted the system. But the adults downloading it from the store 95% of our audience used it just like another educational app designed as a simple pick and chose tool.
Data doesn’t lie, so our great intelligent design was just not accepted by the users. We had to change.
Solution Through Loop Changes
- Option to disable alphabet loop: An option for parents in the parents dashboard to disable alphabet loop. This changed the progression so if you play A in one game a lot you moved to B in the same game.
- Drop down alphabet selector: Users could now select the alphabet they wanted to play by the alphabet menu in the start of every game and in game.
- Non phonetics progression: A linear loop was added so parents who want to teach their children alphabets the traditional way can take advantage of that.
- Progression separation: The progression of each game was separated from the loop so you can be at Q in one game and A on another.
Parents Teaching Behavior Identification
We had two users of our apps, the parent who chooses and downloads the app, pays for it. And the kid who plays with it. So the first on boarding process was for the parent, the games part for the kids. Parents can check in on the progress of their kids using our intelligent dashboard our main app focus.
But from our dashboard usage data, turns out parents love the idea of the dashboard so they buy the app. But they never use it for all practical purposes.
The parents who are involved in the educational process, as defined in the previous section want more control over how they want to teach their kids.
Parents dashboard
Parents were given more access and control on the applications. They could choose their curriculum and teaching plan. Leave the kid to play anything or turn on gamification. Have them teach on common core standards or in the traditional process. Moreover, the progress cards were emailed to the parents.
Learning & Insights
- Teachers are more open towards new teaching methods than parents.
- Children teaching apps are still considered games more than a complete curriculum, it will take more time to change it. Until then have a fallback to traditional teaching methods.
- The more toys you give kids the bigger the mess would be. Control what they consume.
- Children understand visual progression at an early age don’t assume anything about children until tested.
- A/B test everything even the solutions that you’re sure off.
- Push one fix at one time. Don’t update app in a large chunks.
