Duolingo — A User Retention Case Study

Jensen Loke
Mar 19, 2018 · 12 min read
The familiar owl we all know!

Summary

Duolingo is a freemium language-learning platform that includes a language-learning website and the app has about 200 million registered users across the world. I was recently given a case on Feb’18 on how they can potentially increase user retention, and thinking about ways to achieve measurable goals. This article describes in great detail one of the ideas that I’d like to talk about. I’m happy to open source my concepts and get feedback from anyone in the comment section! If you are generally interested is to use competitive gaming to drive user retention, some feedback will be great!

For anyone who’ve used the app before, even though the course materials are engaging, it is still a daily struggle to continually use it without a real motivation; perhaps to master English or learn a foreign language you’ll be visiting. Competitive gaming can make things more interesting by using a peer network to drive engagement rates.

This new feature — aptly named Duogames, will be embedded within the most popular language instruction app in the app store. Duogames is a suite of fun and interactive microgames that allow users to complete and/or issue challenges to one another.

I will be discussing about a specific microgame called Speeduo, the first of these microgames, designed after practice courses that can be completed in 30 seconds or less via character matches or text matches. This ensures a minimal learning curve to drive user engagement and retention. These challenges can be completed by the user and issued to friends or someone within language community who has cleared a similar level.

The rationale behind microgames as a new feature, how these games would work, why these features proposed would work, technical implementation considerations, early hypothesis validation, success criteria, mock customer reviews and future work are topics that will be discussed in detail. An FAQ for both internal stakeholders and potential new users that might help to clarify some doubts over the feature concept will be included at the end of the article.

1.0 Microgames as a New Feature

Microgames use the following social graphs with a friend to friend — node to node connection that is self reinforcing. They help drive user retention by leveraging the micro communities within the larger social network as prescribed in (Figures 1 & 2).

Figure 1, Source: https://linkurio.us/blog/how-to-visualize-your-facebook-network/
Figure 2, Source: https://linkurio.us/blog/how-to-visualize-your-facebook-network/

In Appendix 2, I compiled a list of similar styled gaming apps that Duolingo can adopt potential features from for “microgame” inspirations. These games are similar in several key aspects which Duolingo can adopt. Using the example of words with friends: The app is a simple, decluttered experience without extensive features; they made the UI simple to use, and minimized the learning curve where possible — the most important factor is that users can boot up the app and start it in seconds.

These effects further capitalized the social graph by promoting closeness centrality, described above by combining multiple sharing options. This displays social proof; encourages users to play the game by letting them see how many people are currently playing and what they are playing. The ability to start games with people on contact list (or facebook friends), or even strangers can further help drive user retention metrics.

Microgames can further augment the platform economy established by Duolingo. It creates another mode of interaction that extends beyond existing language club and creates other opportunities to earn both experience points (exp) and Lingots (the in game currency) that can be used to make the overall app experience more enjoyable.

2.0 How these Games Work — Speeduo

Speeduo is a simple game that uses the existing level structures in-app to compile a time based challenge. Users are timed on how quickly they match words in the language of instruction with the words in the language to be learned. They are given the option to follow all of their friends when they onboard onto Duolingo to source for potential competitors and to populate the leaderboard.

Users can issue a challenge to any user and an appropriate level challenge will be assigned. For example, a level 3 user issues a challenge to a level 2 user, a level 2 challenge will be sent to the level 2 user. Because of the skill difference, the level 3 user will earn lesser exp.

Competitors have 24 hours to complete the challenge issued (not real time). Users can also simply play the games themselves to climb the leaderboard. The leaderboard displays the results of the timed challenge and number of victories against fellow users. Highly proficient users with the fastest clearing times and most number of victories are awarded the language level mayor badge which can then be shared or posted on social media. They can also issue challenges to friends who are not on Duolingo via social media (i.e. Facebook) to invite them to download the app and join the leaderboard. The layout and a mockup of the overall game experience is provided in appendix 2.

2.1 Why these Feature Work

Users are used to doing these matches in the language instruction course (minimize learning curve). The UI is kept simple — after completing the level, users are now presented with the option to challenge friends regardless of their levels while simultaneously showing off their own language mastery. Users who have not completed the same level are now incentivized to both complete and compete. The hypothesis is that this becomes the key driver to increase engagement and retention for both new and existing users.

The leaderboard functions as a user feed, displays the progress of users, challenges issued, and creates an additional avenue to showcase language mastery. By not allowing these challenges to happen in real time, we also give users the flexibility to complete the challenges at their convenience within the 24 hours time limit. The hypothesis is that the imposed 24 hours time limit serves the added purpose of driving users to complete the levels within a fixed time frame. We will explore based on usage times if this needs to be extended to 48 hours.

3.0 Technical Implementation Considerations

The speed challenge is an extensible feature — it minimizes engineering constraints by utilizing existing language data resources, adding the timer component. Many elements of the UI can also be reused to implement these features. Since Duolingo is largely focused on mobile (77%), the pilot should be done on a subset of users on mobile.

4.0 Early Hypothesis Validation

Due diligence: since we are basing the first microgames off Practice — Personalized Lessons, we can perform a due diligence by first checking the usage rates of these practices. Some of the key questions to answer will include how often do people use the practice feature, the average completion rate, average time taken to complete (helpful in Speeduo game design).

The alternative way to test for receptivity is to create prototypes that are interactive, using tools like invision, flinto, principle, or framer to test it with users similar to the target profile. These tools can simulate the intended interactions and functions. Additionally, we should get early feedback in a focus group setting. These tests will allow us to quickly determine if the product is viable and get crucial feedback on important elements to focus on for the feature design.

5.0 Success Criteria and Mock Customer Reviews

There will be two components of measuring success. The first is quantitative and the second is qualitative. From the quantitative aspect, these are the following metrics that I would be most concerned about.

Here, we discuss about existing benchmarks and potential increase in retention rates. I estimate success criteria metrics using current user statistics compiled as of Nov 2017, Duolingo users has 25MM monthly users and they spend an average of 10 minutes per day in the app and a majority of them, 53% study English. We compare it against Zygna’s Words With Friends (as of May 17 2016), with 15.4MM MAUs achieving 4.7 hours per month that equates to 9.4 minutes.

Apart from measuring user penetration (user receptivity toward the microgames), these statistics then can help us create benchmarks on increase in retention rates for these users who do play these games. Without doing any form of user testings, I stipulate the stretched target to be a simple addition of the engagement time spent at 19.4 minutes per day per user (at best case) and probably somewhere in between 10 to 15 minutes for a reasonable target to achieve.

From the qualitative aspect, we can use surveys to measure user satisfaction. After completing the microgames, we can launch a web survey via a web browser to capture direct feedback. App store reviews should be avoided at a beta test stage. Alternatively, we can also collect internal feedback or from a select group of beta testers. Mock customer reviews should look like: “Apart from being the best language learning app, Duolingo now allows me to play fun games with my friends and family”, “These new games are so fun and simple!” “Duolingo should launch more of these games”.

7.0 Future Work

The vision describes the end state of the microgames — Duogames as a platform to drive user interaction and engagement. The games provide an alternate mode of engagement with the purpose of making education fun.

The strategy to achieve that vision is to launch a series of microgames, after the successful launch of the first game as a educational gaming platform within Duolingo to expand offerings that encourage user retention and engagement.

Other methods of approach can take on the form of launching a separate app that links between the main Duolingo app. Should the first game be successful, we can consider adopting concepts similar to QuizUp, Words With Friends and DrawSomething using our language courses as the driver content.

Appendix 1 — Frequently Asked Questions

Internal FAQ

Q1) What’s the best type of social graph for Duolingo?

Refer to section 1.0.

Q2) What would encourage users to engage socially at first?

I proposed a “follow all” button where users can add facebook friends. Additionally, the option to follow all friends can be added before the challenge is issued to a friend (Screen 1.3 — Appendix 3) for the first time to encourage maximum social engagement.

Q3) How would you ensure that users want to keep coming back to the social experience?

In the feature set proposed, we make the game activity feed easy for the user to track the challenges they are issued by their friends for continual engagement. Push notifications can be sent to the user when challenges are issued by their friends or just before the challenge expires. We must also keep in mind not to flood the user with too many notifications, accounting for the ones we do send to remind the user to do their lessons.

Q4) What would make posts more exciting to create or to engage with?

In the game status feeds, we allow users to comment directly below and add reactions, much like how these posts are already displayed in the language club section of the app. These posts are intentionally placed as a continuous screen motion under the Speeduo tab, to make it easy for users to interact and engage with them.

Q5) Should collaboration and/or competition play a role in Duolingo’s social experience? If yes, how so?

Yes. The whole notion of the gaming feature with friends allows for primarily competition and secondary collaboration to play a role in Duolingo’s social experience. This article explains the objectives and methodologies behind it.

Q6) How might the in-app experience integrate with our Duolingo real-life meet up events?

The challenge feature can allow new found friends in real-life meetup events to start interacting with one another and break the ice! Once a new facebook friend is detected, the user can now issue challenges and discuss their learnings over a friendly challenge.

Q7) How might the social experience be woven in with the learning experience in the app?

These games, intended to be micro and simple to understand will blend the social experience directly in the app. Some friendly competition amongst peers can boost and incentivize the weaker motivated players to keep going.

Q8) Bonus: Before deciding to invest the design and engineering resources necessary to build your proposed solution completely, how would you validate that your hypotheses driving your solution are correct?

Please refer to section 4.0

Q9) Bonus: Once the changes are developed and released to our users, what user actions would you keep an eye on? What analyses would you do with that data?

Please refer to section 6.0.

External FAQ

Q1) Can I invite my friends on social media or contact list?

For this experiment, we are limiting it to Facebook as this should give us significant reach required to reach our desired engagement rates. In the future, we can implement this for language club learners or access the user’s phone book directly to issue challenges.

Q2) If my friend has not completed the same stage (for e.g. alphabet 2), can I issue him a challenge?

The issued challenge takes on the form of a personalized language lesson. These lessons are only accessible if the user has already completed a level within the app (for example: alphabet for Korean).

This would potentially encourage new users to complete a basic level to participate in the challenge.

Q3) How do I earn exp or lingots?

Completing an exercise now earns you 10 experience points (exp). Later on, experiences can be awarded based on a difficulty mechanism. For a player that wins the challenge, he gets 10 exp. Else if he loses he gets 0. For the beta, we will pilot with exp and explore the dynamics behind how including the lingots will enhance the microgame experience.

Q4) When I issue a challenge to a friend, must it be for the same language?

Yes. We want to create meaningful retention statistics and the hypothesis is that social interaction is maximized when the issued challenge language is the same. Friends will have more learnings to share, and will have a shared interest to talk about and this is the bigger goal to achieve for Duolingo.

Appendix 3 — What is Changing?

The first modification adds a follow all button to the existing facebook friends list. Facebook Friends Screen has a noticeable follow all button to the top of the selection list. This encourages users to add all their friends to capitalize on the friend to friend social graph discussed in the article. This modification allows for users to find other users to challenge on Speeduo easily, thus simplifying the peer to peer gaming process.

The second modification adds a new column to the existing leaderboard page, recording the number of victories as a result from Speeduo or future microgame challenges.

The following screens illustrate the new flow added for Speeduo.

Screen 1.0

Screen 1.0

The practice page accessed via the tab will have a few changes. The page is now of extended length format.

Under strengthen your skills, Speeduo is now directly added below.

Users can easily start a new challenge by themselves or against other users. See the next screen (Screen 1.1) for more details.

The feed below the challenge screen displays both challenges the user issues to other players and the amount of time past on the top right hand corner. As an example, the user has issued a challenge to arubuto 7 hours ago.

Challenges issued to the user are populated, and this feed happens in a chronological sequence.

Expired challenges are shown, since we allow for users 24 hours to complete the challenge, the proposed retention is 7 days, or maximum 20 posts whichever is shorter.

The practice icon has also been changed from a weight lifting icon to a game icon. This breeds the fun notion we are trying to establish with launch of microgames that serve a dual purpose of strengthening the skills of players.

Screen 1.1

When users tap on start challenge, they are brought to the main activity screen — a personalized skill level practice session with the added component of a stopwatch at the top right corner of the screen.

Screen 1.2

Clicking on continue brings the user to the next screen (Screen 1.3).

Screen 1.3

When issuing a challenge to a friend, the individual names can be highlighted and the user can select a maximum of five friends to send the challenge to. They can only send a maximum of one challenge to each friend at any one time so as not to spam their friend’s inbox.

A pop up will show that the challenges are successfully issued and the user is brought back to Screen 1.1.

Appendix 4 — References

https://www.nytimes.com/2017/03/18/style/addicted-to-online-games-words-with-friends-prolific.html

https://linkurio.us/blog/how-to-visualize-your-facebook-network/

https://www.fastcompany.com/1810607/how-words-friends-beat-scrabble-its-own-game

User metrics research

https://www.google.com.sg/amp/s/expandedramblings.com/index.php/duolingo-facts-statistics/amp/

http://www.vertoanalytics.com/consumers-spend-1-billion-hours-month-playing-mobile-games/

Jensen Loke

Written by

Technical Product Management @temasek digital tech| Building AI & big data products #rootaccess

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade