Self-learning about Data Analytics as a UX Designer: #1 Don’t Underestimate the Art of UX

The value of data nowadays is as significant as oil. As a UX designer, data is also an insight; UX can no longer rely solely on results from one source, such as the qualitative approach. Currently, it is more impactful when you gather quantitative insights and share them with stakeholders. An easier way to convince key stakeholders is backed up by the sample size.

My Journey Before Analytics

I have two reasons to start learning more seriously about data analytics. This is more like the story of a rookie who tries to experiment and fails a bunch of times. For those people who a good tips and tricks feel free to skip! If you would like to have a companion on your journey as a rookie in this field, we are the same!

1. Good things, even if you have a chance to work with a company that values data, collects, and tracks it. But once you would like to start, as a person who is not familiar with this, I find it so challenging and requires a steep learning curve to understand the key concepts and apply them to daily work
2. Also, for a person working in a non-product company, the use case that I find is not only about your skill set but also that data on the client side is not clean or well enough to use. Or they don’t even track data that is useful enough for analysis.

Note:

Also if you wanna skip to How can I start to track feel free to go to the next chapter.

Self-learning about data analytics as a UX designer #2 Start from scratch

My Learning Objectives

I have a few simple objectives, all aimed at ensuring that I can embark on my journey without the fear of making things wrong.

1. Become familiar with applying data analytics for insights.
2. Get hands dirty with data tracking and understand the workflow between designers, data scientists, and, in some cases, developers.
3. Inspect and adapt to real challenges in situational projects.

Project Context

Before we start, I would like to express my gratitude to the ‘เรียนวิธีคิด ผ่านวิธีโค้ด’ team (‘Learn how to think through the coding method’ team) for allowing me to use their data as my use case and for collaborating with me thus far on event tracking.

About the ‘เรียนวิธีคิด ผ่านวิธีโค้ด’ team (‘Learn how to think through the coding method’ team):

The game startup offers a systematic thinking process through learning on a coding platform. Their mission is to enhance the skill set of Thai children by incorporating the basic skill set of logic in thought processes.

Their current product is a freemium game, including both game series and mini-games that users can complete in one sitting. They are targeting children from early primary school to early high school.

Their problem statement

They have already begun tracking the performance of the general website. While there is general information available to gauge overall monthly engagement, when it comes to more specific game performance metrics such as user engagement duration or identifying the level at which users tend to drop off in the game, they lack sufficient insights for justification.

3 Approaches to Get Started: Don’t Underestimate the Art of UX

When working on a project from scratch, it’s common to have nothing ready, and it’s important not to spend too much time setting up everything perfectly. To get started efficiently, it’s essential to ensure a good understanding of both the business context and insights, as I’ve found that this is how a UX skill set can close gaps effectively.

Stakeholder Interview: Conduct interviews with stakeholders to prioritize which parts of the website should be the focus.

Usability Test: Observe the existing product and formulate hypotheses.

Analyze the performance of the existing product: To understand our current position.

Key insight from stakeholder interview

I would be heading in the wrong direction from the start if I didn’t discuss with the ‘เรียนวิธีคิด ผ่านวิธีโค้ด’ team (‘Learn how to think through the coding method’ team). If I start by looking at analytics first, I might provide them with incorrect suggestions.

#1 Insight — Game as a Medium for School Subjects:

They are in partnership with schools, and the product positioning involves incorporating the subject ‘Computing Science’ into school media. This means that the primary users are not just general children interested in learning how to code but also teachers who teach them and children in schools.

#2 Insight — Mini Game is the Most Used:

The mini-game is designed for users to complete in one go, making it an ideal exercise for students to finish within the subject’s hour. Teachers then set KPIs for students to obtain scores, requiring them to complete the game and receive the game’s certification.

Key insight from the analysis of current product performance

#1 Insight —Significant loss of user retention from week 0 to week 1:

There is a drop from 8,600 users (average) in week 0 to 700 users (average) in week 1. However, from the second week onwards, user retention stabilizes at 280 users (average).

Assumptions:

1. The main goal for users is to obtain certification only once and not continue.
2. There are not enough games or methodologies to attract users and encourage them to continue after acquiring first-time users (e.g., a leaderboard).

#2 Insight — Ramakien mini-game is the peak avg. engagement time:

There are a total of 4 mini-games, with 3 of them being featured as the top-engaging. The other top engagements include the homepage, typically navigated often by users, and the page for downloading media.

Currently, the most engaging mini-game is Ramakien at 22 Mins

Questions:

1. Why is the engagement time for Ramakien 22 minutes, higher than the other mini-games?

Based on a study of terms of engagement, this is more likely related to how users interact with the game, such as clicks or shares, which in this case could indicate the total time users spend playing the game.

After discussions with the team, they predicted, during the game’s launch, that the benchmark for users to complete each mini-game should be ‘only a maximum of 15 minutes’.

Key insight from the usability test

Comparing the before and after usability testing with the current product, I discovered many deeper hypotheses that are not just about measuring the success of users who complete the game. Usability, in this case, incorporates insights from two different resources:

1. My usability tests were conducted with real children.
2. The experiences of the team during tours and interactions with school users.

Online usability test with children

#1 Insight — Standalone vs. Social Competition:

Users who play alone are less likely to resume the game after a Game Over compared to users who play with a group of friends. Children are more easily encouraged by social competition because, somehow, they enjoy bragging to each other.

#2 Insight — High Chance to Quit for Failing at the Same Level Repeatedly: Users who experience a Game Over at the same level are more likely to quit the game compared to users who fail at different levels.

The #2 insight inspired me to derive a key insight:

“The loss of progression is a key factor that motivates users to quit the game.”

Questions:

Will users who progress to later levels find it easier to give up after a Game Over? After a Game Over, users will be redirected to the 1st level, which might be discouraging for them.

My first chapter is done, but there is much more to talk about in the next chapter. Let me recap this exploration phase.

Recaps My Learning in 1st Chapter:

1. Looking at an overview and connecting the dots:

Let’s not rely on a single source of truth. Start by gathering insights from various channels and connecting the dots. Don’t focus only on the number of dashboards.

2. We can start small:

At the starting point, I felt that dealing with things from scratch was quite challenging. How could I gain good insights from nothing? However, insights are everywhere. Just by discussing with business stakeholders and conducting usability tests, I could get clues to get started. If I wait until tracking data analytics finishes, it would consume time and yield nothing.

3. Collecting insights from usability tests can guide the hypothesis:

It sparked an idea while observing children playing around the games. It gave me different perspectives that I hadn’t considered while brainstorming hypotheses without this observation, such as how much patience children have until the end of the game.

Next Chapter

In the next chapter, I will delve into how I can apply these hypotheses for data analytics tracking and discuss the challenges I encountered while working on new things.

If you’re ready, feel free to continue!

Chapter 2:

Self-learning about data analytics as a UX designer #2 Start from scratch

Chapter 3:

Self-Learning Data Analytics as a UX Designer: #3 Let’s do the most fun part! Summary a thing.

Big thanks to all my supporters

Thank you for your encouragement, guidance, and valuable suggestions: Pete Chemsripong, Pee Tankulrat

Thanks for the valuable opportunity on playground data sources and your collaboration effort: S.Songklod IToon, Tann Hiranyawech