Which User Research Method Is Best for Mobile Apps?

No two software products are exactly alike, even among similar products aimed at similar markets. This is what makes user research such an exciting process-not to mention the practically infinite ways in which users can interact and engage with the products we design.

This becomes especially important when designing and developing mobile apps.

Although every mobile app is different, conducting user research to evaluate how our users interact with our mobile apps is similar, regardless of an app’s intended audience or function. When it comes to conducting user research for mobile apps, there are several qualitative research methodologies we can use.

In this post, we’ll be taking a look at three such research methods, including their pros and cons.

By the time you’re done reading this post, you’ll have a much better idea of the advantages and disadvantages of each approach, and the situations to which each method is best suited.

What Types of Qualitative Research Methods Are Well Suited for Mobile Apps?

Before we dive into the strengths and weaknesses of each research method, let’s take a moment to outline each of the three methods we’ll be looking at:

• Journal studies: Sometimes known as diary studies, this research methodology asks participants to keep a diary or journal of how they use a mobile application. Journal studies are great for identifying when users are most likely to use an app and in what context.
• Laboratory studies: Lab studies allow user researchers to observe a user interacting with an application in a carefully monitored laboratory environment. Researchers might ask users to perform specific tasks using an app but can also simply observe how participants use an app on their own.
• Observational studies: Similar in structure to laboratory studies, observational studies also involve directly monitoring research participants as they use an application. What makes observational studies distinct from their lab counterparts is that participants are often recorded as part of an observational study for further review and comparison purposes.

Now we know a bit more about the three types of qualitative user research we’ll be looking at, let’s examine each methodology in a little more depth.

Mobile App User Research: Journal Studies

When it comes to conducting qualitative user research, journal studies can be immensely valuable.

The primary purpose of journal studies is to provide researchers with a record of when users used an app and in what context. That makes journal studies an excellent way to evaluate the frequency with which users engage with an app, as well as the specific circumstances under which users open the app in the first place.

One of the reasons why journal studies can be so insightful for user researchers is that they allow users to record their thoughts about the experience of using an application in real time; they don’t have to remember what using the app felt like or rely on overly broad generalizations that may be of less use to research teams.

Journal studies do have one glaring drawback over which researchers have virtually no control: making sure research participants note when and how they used the app. Even users with the best of intentions may inadvertently forget to keep up with their usage journals, which can call the reliability of the data into question.

Mobile App User Research: Laboratory Studies

Many user researchers love conducting lab studies, and for good reason.

Lab studies allow user researchers to create specialized environments in which users’ emotional responses to using an app can be monitored and evaluated. This type of qualitative research method also makes it easy to detect subtle visual cues that reveal users’ underlying emotions, such as frustration or excitement.

One of the most useful aspects of laboratory studies is that researchers can ask participants follow-up questions as the users interact with the app. This provides a great deal more contextual information about precisely what users found enjoyable or frustrating about a software product, as well as greater insight into potential roadblocks or bottlenecks in an app’s flow.

The main disadvantage of laboratory studies is that, unfortunately, lab environments bear very little resemblance to the circumstances in which users will actually engage with a product. This can often result in a significant disconnect between perceptions of an app’s functionality and the realities of using the app in a real-world situation.

There’s also the issue of bias to consider. Some research candidates may want to “please” the researchers, which will affect how and why users perform certain actions in an app or respond in a certain way. This phenomenon is sometimes called the Hawthorne effect. This can skew the research data itself, which, in turn, can lead to flawed assumptions.

Mobile App User Research: Observational Studies

Similar to journal studies, observational studies also seek to answer questions about when and how users engage with an application. The primary difference between the two methodologies is that observational studies are typically recorded for further analysis.

Observational studies are ideal for making the research process as easy as possible for participating users. The fact that observational research studies are usually recorded means users and researchers alike can focus on the experience of using a software product rather than interrupt that experience to record their thoughts and observations. But as useful as observational studies can be, they aren’t without their downsides.

For one, observational studies can be intrusive, even when recorded using small, inconspicuous recording devices. And, as you might remember from physics class, the very act of observing a phenomenon fundamentally changes the phenomenon being observed-a theory known as the observer effect.

Another problem with observational studies is that the participants who agree to be recorded when using your product may not necessarily be representative of your app’s target market or actual user base. These discrepancies can be difficult to reconcile, which can make observational studies as problematic as they can be useful.

Which User Research Method Is Best Suited for Mobile Apps?

So, which of these three qualitative user-research methodologies should you use when developing your next mobile application?

All of them.

Since each of the three research methodologies above has distinct advantages and disadvantages, the question isn’t which method is best-it’s when to use each separate methodology.

For example, you might want to begin your research process with a journal study to identify some initial data points before utilizing a laboratory study to see whether the user responds similarly in a controlled environment.

It’s also worth considering exactly what you’re testing before settling on a particular methodology. For instance, an observational study may be the right choice if you’re trying to gauge the effect of a brand-new onboarding flow or major new design update. Conversely, laboratory studies may be of less value if you’re primarily concerned with the situational context of how your users interact with your app.

Finally, it’s vital to consider user research in the context of the purpose of your app. Users will interact with a productivity tool much differently than they’ll interact with a navigational app, for example. Another factor that should inform your research workflow is whether or not your mobile app has been released yet. Initial user research for a product that’s still in development will be very different from ongoing user research based on actual usage data generated by real users, so it’s important to account for these variables as best you can when considering various research methodologies.

Only you can decide which of these qualitative methodologies will meet your needs. To ensure that you’re getting the user data you need, it may also be worth combining one or more of the qualitative methodologies above with quantitative user data to prove or disprove your hypotheses.

Originally published at https://blog.getenjoyhq.com on April 16, 2019.