Decoding Duolingo: A Case Study on the Impact of Gamification on the User Experience
Overview
The gamification strategy employed by Duolingo has numerous benefits, including better absorption of knowledge, increased attention and motivation, and the promotion of habitual practice, among others.
However, we have observed a downside to gamification that manifests as the heterogeneous effectiveness of Duolingo as a learning tool, resulting in pain points for some users in their experience.
Role: UX Researcher
Timeline: Jul — Aug (2 months)
Product type: Free version of the mobile application
The Process
- Problem statement
- Research questions
- Hypothesis
- Objectives
- Target users
- Secondary research
- Primary research
- Triangulation of results
- Opportunities
What is the main problem?
Applying game mechanics to language learning may create certain issues:
- Distractions from the main learning goal
- Negative feelings arise from ignoring different learning rhythms and personalities that are not competitive.
Research Questions
We guide the research by defining the uncertainties that we will seek to answer.
- What are the feelings caused by gamification?
- What impact does gamification have on behaviour?
- Are there frustrations associated with gamification?
- How can we achieve a better learning experience?
What hypotheses are we starting with?
We define assumptions that we will try to validate or refute through research.
General Objective
Investigate the user experience with gamification in Duolingo.
Specific Objectives
- Understand the sensations produced when using Duolingo.
- Discover the moments of the gamified experience that generate the greatest frustrations.
- Identify the consequences of frustration in behaviour with the application.
- Understand the expectations and objectives that users have with Duolingo.
- Understand the most common and preferred way of using the application.
Who are we targeting?
- Users from Argentina
- Aged between 20 and 40 years
- Who have used the Duolingo application at least three times a week since 2019.
Research Methods
We define the techniques and tentative sample to begin shaping the research.
SECONDARY RESEARCH
We read, group, and analyze reviews on the App Store, Play Store, social media, and blogs in search of patterns. Then we analyze all the moments of the experience where gamification is applied.
- User feedback and voice
- Analysis of gamification in the application
PRIMARY RESEARCH
Exploratory: We use different techniques to gain a better understanding of the user and the problem.
- Interviews (5 users)
- Participatory activity
- Supplementary Surveys
Evaluative: We will seek to evaluate solutions to detect problems and possible improvements in time.
- Concept Testing (6 users)
What do users think?
We gather and analyze user feedback from various sources with two objectives:
- Identify elements that have the most impact on user satisfaction and are related in some way to gamification concepts.
- Find patterns to better understand motivators, frustrations, when they occur, their causes, and consequences on behaviour.
Findings
We have observed that certain gamification characteristics create anxiety, pressure, and frustration among users.
- The life system makes the learning process feel truncated.
- The competition in leagues distracts from the main learning objective.
- Notifications are perceived as frequent and annoying.
- The use of the application is intermittent.
- Although leagues are an incentive for completing lessons, they also create pressure to win or even remain in a division, especially in the higher levels.
- Streaks are a significant motivator for users.
Gamification Analysis
- Duolingo promotes competition through leagues and tournaments.
- Users earn medals, experience points, and gems as they achieve daily and monthly goals.
- Courses are divided into thematic units with different levels of lessons and stories.
- Users earn experience points for each lesson or story completed and lose a life for each mistake.
- The system recovers a set of 5 lives after a few hours.
- Duolingo encourages daily language practice through the “streaks” metric and associated notifications.
- The app borrows concepts from games such as “experience,” “reward,” and “gems.”
User Interviews
Goal: To understand the feelings and identify frustrations in the experience with Duolingo.
Sample: 5 users
Duration: 30 minutes
Structure: Semi-structured
Mode: Remote
Type: Moderated
Roles: 1 Moderator 1 Observer
The questionnaire focused on:
- Language learning preferences
- Motivations
- Usage habits
- Frustrations with the app
- Leagues and divisions
- Life system
User Personas
Based on the information gathered from the interviews, we developed more detailed user personas.
Talking to users helped us discover key insights
Social interaction is critical for improvement
Even those who prefer to learn a language individually highlight the importance of social interaction to practice and improve.
Both teachers and peers are valid options, although practising with native speakers is ideal.
This is even more important when it comes to reaching high levels of language proficiency.
Consistency is necessary to learn
Users believe that sustained language practice is the best way to learn, and Duolingo helps them maintain consistency in various ways.
- The repetition-based lessons help to fix concepts.
- The gamified experience makes language learning more entertaining. Elements such as streaks and leagues create a sense of competition and accomplishment.
- The short lessons feel like mini-classes that users can complete in just a few minutes a day. This aligns with users’ tendency to use the app during idle moments throughout the day.
Objectives and motivations with Duolingo
Some users have goals that go beyond language learning, such as applying the language in their personal or professional lives, for example in a trip or while doing paperwork to acquire a new nationality.
Others, on the other hand, are learning in the medium or long term and use Duolingo as a complement to other methods.
What leads to interrupting the use of the app
There are various reasons why users stop using the application, at least for a while:
- Boredom due to the repetition of lessons
- Lack of challenges
- Stress due to competitiveness
- Loss of initial motivation to learn a language
The life system is demotivating
The life system is seen as a nuisance and a means to promote the premium version. Additionally, waiting for hours to recharge lives and continue playing is demotivating.
The only positive aspect is that it can help pay more attention to avoid making mistakes. That’s why most users don’t propose eliminating the system but modifying it.
Competitiveness in Leagues and Divisions
The competitiveness generated in this system can be motivating or stressful.
The former occurs when two known people compete, helping to maintain practice consistency. The latter occurs when many notifications are received, and the person feels pressure to stay in the top positions in the divisions or at least not to descend.
Empathizing with users
Why did we decide to create an Empathy Map?
The goal is to delve deeper into attitudes, behaviours, and contexts to create a more complete user profile. This way, we can generate ideas that better meet needs and expectations.
INPUT
- Participatory design activity
- Digital surveys
- Interviews
- Voice analysis and feedback
- User Persona
Concept Testing
It seeks to evaluate the likelihood of success of proposed solutions to optimize design and development efforts.
Testing characteristics
Testing concepts
The concepts were well-received
Tested concept
Requesting and sending lives to friends.
Pain Point
Waiting for hours to recover the full set of lives is demotivating for users and distracts them from their lessons.
Proposed Solution
Being able to send and receive lives between friends could not only accelerate the recovery time but also strengthen the sense of community.
More than 80% of the participants have at least a good impression of this proposed functionality, while for 33% it is excellent.
67% of the participants are likely or very likely to use this functionality, while 34% consider it less likely.
Tested concept
Competition mode with friends and practice mode.
Pain Point
Competing with strangers distracts from the learning objective and can provoke negative feelings.
Proposed Solution
Choosing between a competition mode with friends and a practice mode would add to the personalization of the experience.
For 100% of the participants, their first impression of these new competition modes is at least good and for 50% it is even excellent.
67% of the participants are likely or very likely to compete with friends, while 34% consider it less likely.
66% of the participants are very likely to want to use the practice mode, while 33% consider it somewhat likely.
Research Triangulation
We organized and verified the results obtained with each method to increase their confidence.
The current life system is demotivating.
The life system is considered one of the most negative aspects, as it is perceived as a nuisance and a means to promote the premium version.
Waiting for hours to recharge lives and continue playing becomes boring, demotivating, and distracts users who feel that their learning process is being interrupted.
The only positive aspect detected is that it can help users pay more attention to avoid making mistakes.
The concept of receiving and sending lives to friends received high acceptance among users. In this way, the sense of community is also strengthened.
Associated Hypothesis:
- Losing lives for each mistake made during lessons generates frustration.
Competition generates both positive and negative feelings, depending on the user.
Leagues are fun and motivating, especially when competing against known people, helping to maintain the necessary practice consistency.
However, when competing against strangers, feelings change and tend to be negative: pressure, exhaustion, and boredom. This is emphasized when notifications are received about changes in position in the Division, and the person feels pressured to maintain it.
Opinions on the influence of the Leagues on the learning objective are also diverse: while some find them motivating, others are distracted from the main goal. There is also a third, minority group that remains indifferent to the Leagues.
We have detected opportunities for improvement with proven high acceptance for each user to choose the level of competition that they believe is best.
Associated Hypothesis:
- Users feel that sometimes they play just to maintain their position in the Leagues and not to learn. VALIDATED
2. Users would be more satisfied if they could use Duolingo in a less competitive, more relaxed way, at their own pace. VALIDATED
3. Leagues generate anxiety by always being in competition. VALIDATED WITH RESERVATIONS
Gamification does not have a significant negative impact on usage frequency and dropout rates.
Users believe that their experience on Duolingo is mostly entertaining because it feels like a game, which leads them to feel incentivized to use the app more and more.
On the other hand, while competition or the life system may demotivate app usage, they do not have a significant impact on it.
The factors that most affect usage rates are:
- Personal goals for learning a language
- Lack of time
- Loss of interest
- Boredom from repetition of vocabulary and exercises
- Lack of challenges
Associated Hypothesis:
- Gamification produces frustration and/or anxiety in some users. REFUTED
What were our findings?
Federico
25 years | Buenos Aires
He is learning English to better communicate on a trip with two friends. He wants to learn quickly because they are travelling in a few months. He needs to be able to recover lives quickly because he makes many mistakes as a beginner and feels his learning is quickly cut short.
Soledad
34 years | Cordoba
She is learning French because she enjoys French culture. She already has a foundation in the language and uses Duolingo for review. She needs to be able to personalize her competition in the Leagues because she wants to learn at her own pace and without pressure, as it is only a hobby for her.
Opportunities
Throughout the research, we found various opportunities for improvement to investigate to continue improving the experience.
1- How to avoid the experience of feeling monotonous with continued use of the application?
Retention strategies: working on the current attractive features and motivations found in Duolingo to reduce the implementation effort. In the medium term, we could analyze innovations that maintain high expectations and interest.
2- How to personalize the learning experience to be challenging and flexible enough, without going against business goals?
Those who have some knowledge of the language find the path of units to follow unchallenging and would prefer to choose the optimal order according to their needs.
It is important to note that the functionality of choosing any unit on the path, regardless of progress already made, is a premium feature. We must then ask ourselves how we can provide enough flexibility so that users feel their experience is challenging without making functionalities that are paid for free.
3- How can we encourage a sense of community?
Social interaction plays a very important role in learning a language. Considering that the problem is that it feels monotonous with continued use, we can ask if we could think of new lessons or exercises that involve more interaction between users.
4- How to personalize the experience according to different usage situations?
The most common usage situations are related to downtime, mainly in transportation, and moments when the user feels they can isolate themselves from their environment.
Considering that two of the most valued features are its practicality and short lessons, we could think of personalizing the experience even more for the main usage situations that may arise. An opportunity to analyze could be to offer mini-games.
Next Steps
- Design a functional prototype of the tested concepts to validate with users.
- Iterate the design.
- Develop and launch the first version.
- Launch a second phase of this research to analyze the reception of the new features.
- Create a feasibility, desirability, and viability matrix to prioritize opportunities.
Thanks for reading!
I’m a UX/UI Designer and this project is part of my UX studies. I really enjoyed this UX Research process and I hope there will be more in the future.
You can also find me on LinkedIn and Behance or contact me at florsoledaddaniele@gmail.com
