Increasing Online Course Completion Rate: a UX Research Study (part 2)

Courtesy of unDraw.

This is the sequel to part one of our research where we figured out the pain and gain points of the Skill Academy mobile app. We jump into the fun part of the research: exploring different ways to increase the completion rate of online courses.

User-centered Research

There are three main methods that we used to conduct the second phase of our research:

1. In-depth Interview
2. Secondary Research
3. Comparative Analysis

In-depth Interview

As we have established the target market of our users, we proceeded to interview five people to know more deeply about why people usually leave their online courses unfinished. The participants’ criteria were:

1. Tech-savvy, uses smartphone frequently
2. Young professional
3. Have experience in enrolling on an online course

We asked them to describe their online learning journey, from discovering to finally completing (or leaving) the course. From that question, we asked additional questions conditionally, such as how they encountered a problem and what they felt about it, or potential improvement suggestions. Each interview took about ten to twenty minutes.

Made using WordItOut.

We collected several recurring keywords and turned them into a word cloud. The bigger the size of the word, the more often the word was mentioned in the interviews.

The oft-recurring words revolve around the features of online courses, such as videos, subtitles, quizzes, interactivity, and lesson progress.

Lesser-spotted interesting words include motivation, fun, laziness, and repetitive. The keywords inform us of the minimum features that might be essential for our app revamp, and some clues about what people feel about online courses and what to do to improve its user experience.

We also believed that the findings were in line with our hypothesis (People tend to not finish their courses due to a lack of motivation).

Secondary Research

Having gathered clues related to the lack of motivation and fun in online courses, we elected to dive into existing research material on the internet to explore different options to increase motivation and engagement of users.

eLearning Industry and Desire2Learn detailed several ways to increase user engagement. Some of the methods have already been implemented in the existing app, such as class progress tracking. There were also methods that can be quickly implemented in our revamp, such as:

1. Keeping things visually simple
2. Choosing the right colors and fonts
3. Providing platform for discussion

Options that we considered but decided against implementing at the time:

1. Learning roadmap, where there is a fixed pathway from beginner classes to advanced classes. We decided not to adopt this because there were too many diverse topics inside Skill Academy with many different teachers, and from the operational point of view it would be hard to build a unified roadmap for all those classes.
2. Learning targets which can be set individually. The above reasoning stands: we were not confident that the existing classes could be divided into learning milestones effectively.
3. Leaderboards, where users can compare their exam scores against each other. Although the concept is interesting, we felt that the available topics were not suited to be turned into a competition format. The classes were aimed at professional expertise such as guide to TOEFL, guide to making CVs, and professional branding principles.

We ultimately chose to implement gamification on our Skill Academy revamp designs. As quoted on one journal article (D. Huynh, et al.):

Gamification is a term that refers to the use of game-based elements such as mechanics, aesthetics, and game thinking in non-game contexts aimed at engaging people, motivating action, enhancing learning and solving problem.

The researchers analyzed gamification in depth, in particular using Duolingo as the main research object. On Duolingo’s use of gamification, they wrote:

As we introduced, Duolingo is a learning platform, which means that it is a serious environment and game elements are used to increase motivation and engagement of learners, they are not used to make a course become entertaining or relaxing as fun games.

As explained above, the main benefit of gamification is how it helps increase motivation and engagement. It is fundamentally different from an actual game, where the aim is to provide entertainment. We believed that by gamifying Skill Academy, we would be able to help people keep using the app and ultimately increase their courses’ completion rate.

To further strengthen our decision, we jotted down a simple SWOT analysis to break down the pros and cons of the concept. We felt that the strengths and opportunities could potentially solve the problem, and outweighed the imposed weaknesses and threats.

By this point, we felt that we were able to turn our hypothesis into a more concrete version, a problem statement that details how we proposed to solve the problem.

Problem Statement:

We can make people complete more online courses by implementing gamification.

Comparative Analysis

We also tried to compare the gamification-related elements of two apps: Ruangguru and Duolingo. Ruangguru is the first app developed by the team behind Skill Academy and focuses on Indonesian school-age students, with a cheerful design and some gamified elements. Duolingo teaches you all kinds of language from all over the world and packages all of its lessons into a game-like system. Both are indirect competitors that target different markets with good gamification systems.

Gamification comparative analysis.

Several common elements that both apps share are the levelling system, reward, and learning target. Ruangguru’s gamification system is called Ruangguru Adventure, with additional focus on customizable user avatar that can be outfitted with different hairstyles and clothes. Duolingo has its own mascot named Duo, an owl mascot that serves as a coach and instructor in different segments of the app.

A missing element that we felt might fit into Skill Academy’s current business model was vouchers. It might not fit into Ruangguru and Duolingo’s subscription model, but giving discount vouchers for further purchase (inside or outside the app) might be beneficial for Skill Academy’s pay-per-lesson model, encouraging multiple buys at a lower price and ultimately increasing user retention rate.

Additionally, an excellent breakdown of Duolingo’s user journey by the growth.design team analyzes a feature that we felt would aid us in solving the problem, namely the interactive elements that capitalizes upon humans’ emotional traits and serve to boost user retention. Several noteworthy elements include lifecycle emails, welcome back reward, streaks, sunk-cost effect, and notifications.

Among these elements, we felt that the welcome back reward would fit our intended gamification system. The sunk-cost effect would also be an interesting addition to the system, where users are required to spend in-app currency to join a challenge, then in turn feel more involved and focused to complete it.

Research Deliverables

After we dived into in-depth research, we made some deliverables that would give us a better picture of our users and could complement our design process.

Persona

First, we built a persona: Budi (Fun fact: “Budi” is the “John Doe” of Indonesian grade schools). Budi is your typical Indonesian employee who is currently seeking ways to upgrade his competencies.

Customer Journey Map

After learning about the journeys of users via in-depth interviews and finding out more about gamification, we compiled all of them into one Customer Journey Map starring Budi. We also included a storyboard to better visualize the journey and give a sense of story narrative.

Made using UXPressia.

Most people will stop their online learning at step 4, and will eventually sink back into routine (steps 5 and 6). With the inclusion of gamification and interactive elements we hope to design an online learning application that helps increase motivation and upgrade their skills eventually, just like Budi.

Conclusion

In summary, these are what we established from the research findings:

• People who drop their online courses halfway usually lack motivation and seeking an element of fun.
• Gamification can help online courses in increasing motivation and retention rate of customers, and in turn increasing completion rate.

Next Steps

Both the user-centered research and the previous usability research were then used as the basis of our revamp of Skill Academy. Next we would start the design process and build the interactive prototype.

Lessons Learned

These lengthy posts tell only a fraction of the original research report, but I have detailed the important parts. Going through the research, I learned several things:

• In-depth research takes time and should ideally be conducted before the actual design sprint process.
• Interviewing users without bias involved is a skill that needs to be practiced over and over.
• Every step of the research process should be logical and help the team move closer to solving the problem.

Kudos to my fellow team members, our instructor, and the rest of the Purwadhika UI/UX Design class for their teamwork and input during the research process.

Update (March 6, 2020): M Abizar Fahri, my teammate and the UX Strategist slash Information Architect in our project, has written an article detailing the design process. You can check all about it, including the designs and the final prototype, right here!

— Home, where the roof gives shelter from outside influences and fake news.

Want to learn more?

If you’d like to…

· get an industry-recognized Course Certificate in Usability Testing

· advance your career

· learn all the details of Usability Testing

· get easy-to-use templates

· learn how to properly quantify the usability of a system/service/product/app/etc

· learn how to communicate the result to your management

… then consider taking the online course Conducting Usability Testing.

If, on the other hand, you want to brush up on the basics of UX and Usability, then consider to take the online course on User Experience. Good luck on your learning journey!