How we helped Burda gain qualitative research data with usability testing

Qualitative user research enables researchers to gain a deep understanding of users’ feelings, motivations, and experiences. This is why companies nowadays put their trust not only in quantitative user research data but also look for qualitative data analysis to support their findings.

The “fold in the rug” experiment / Flattening the “fold in the rug”

A really tangible idea of usability testing and a good user experience is the paradigm of the “fold in the rug”: Let’s imagine a team of researchers learning about a “fold in the rug” that only a small number of passer-bys stumble upon — and the team eliminates the obstacle after this discovery. In consequence, the whole experience is improved, not only for the minority that stumbled before, but as well for the other people that could “cross the room without tripping over the fold in the rug. This improvement of user experience can either be felt consciously as well as unconsciously.

Hence, a big upgrade was introduced for all passers-by — thanks to a small insight from a few of the research data pieces. Providing evidence of how qualitative user testing is key for successful improvement, is the following use case which we conducted for Burda.

Status Quo: The User Experience of einfachbacken.de

Together with Userlytics and with the use of our platform consider.ly, our team conducted a usability test for Burda’s website einfachbacken.de. The website is a curated collection of simple baking recipes, suitable for beginners and those who are already more experienced in the kitchen. Currently, einfachbacken.de receives about 10 million visits per month and takes on the role of a special Burda brand that will be further developed in the future.

The website’s design is bright and friendly. A light color scheme combined with appealing food photography invites users to look for recipes and find baking inspiration. The main menu displays a clear overview of what is provided: There are different categories for the recipes, a baking “news” section, and information about baking accessories. At the top of each recipe page, you will find a photo of the final result, e.g. a chocolate cake, followed by the instructions on how to recreate the baked goods. On the bottom of the page, users are able to comment and rate the recipes using stars on a scale from 1 to 5.

Since Burda didn’t have much experience with qualitative user testing — this kind of study was quite new and unexplored ground. . So far, the company had been concentrating heavily on quantitative research. Hence, Burda did not have a special, designated data storage for UX-related insights. Since Burda wanted to be more prepared for future studies they chose to collaborate with us — using consider.ly to store their research findings.

Working with a strong companion: Userlytics

Our first step was reviewing einfachbacken.de to gain a broad understanding of how the page is structured.

Next, we searched for a companion in our journey for helping Burda: A colleague willing to collaborate with us through recruiting participants for our usability study and its conduction subsequently. Luckily, we found Userlytics and its platform — this way, we were able to offer Burda full-service support:

While Userlytics was responsible for both a fast and targeted adequate participant recruiting and the use of their qualitative user experience testing platform, we at Usertimes focused on the evaluation and documentation of the results — supported by the Userlytics platform as well as our tool consider.ly.

With their proprietary global participant panel, Userlytics makes it possible to find the right demographic. Thanks to its user panel of more than 750,000 users from all over the world, its advanced demographic filters, and customizable screeners, we chose Userlytics as a partner. Because of their advanced UX testing platform and feature set, integration into consider.ly’s work environment was easy.

The Userlytics platform includes a series of very useful features to perform usability tests — such as branching logic, advanced metrics (for example NPS or SUS) to quickly locate key insights and specific capabilities for improvement of information architecture like Card Sorting or Tree Testing. Userlytics also allows you to create compilations of video clips organized around key themes, as well as machine-based transcription in multiple languages. The Userlytics platform is also highly scalable, allowing the addition of more participants to a study via a simple click.

After the testing sessions, we imported the resulting videos to consider.ly and continued working with them, tagging important passages and creating a holistic picture out of them. Enabling us to analyze the usability testing results.

All in all, our team at consider.ly was able to finish screening, crafting the study and its documentation just one week

How usability testing results deliver answers

From our findings, Burda was able to derive important insights on succeeding features of the einfachbacken.de website, as well as possible small changes that could be made immediately.

The product team at Burda was especially indecisive about certain “customer rating features” on einfachbacken.de. The qualitative user testing helped the team to better understand why customers want this kind of feature and what they expect in terms of look and feel or usefulness.

An Example: “Real” 5-star Recipes

On the existing site, every customer could rate the recipes with a rating between 1 and 5 stars. The site then automatically aggregated the ratings and showed them along with the recipe.

The problem: All ratings were rounded up during the aggregation. So, every recipe above 4.4 stars would be displayed as a 5-star rating. Those rounded numbersled to disbelief among the testing participants and shed a negative light on the whole site as a consequence of the lesser brand trust.

The research results helped Burda to learn more about how they are perceived by their customers and give the product management a better understanding of their target audience.

Next steps: Ongoing collaboration

In practice, continuing to work with results sometimes forms an obstacle for companies.

With consider.ly we address this challenge early on: We offer a convenient option of visualizing results. In our tool, the raw data is linked to the results. This has the advantage of being able to answer questions quickly. Having results linked to their proof also supports a better understanding of the conclusions made. Additionally, this way it’s easy to involve a whole team in the process.

Thanks to our usability study, Burda has laid the foundation for the following work. The collected data is further exploitable and could support future evaluations. To make this a reality, it is crucial to keep the knowledge gained actionable. A good way to do so is to save all raw data, notes, and the procured insights in a repository, just like our tool consider.ly offers.

Burda is now in a position where every new study contributes to its ever-growing research repository. Thus, helping the organization better understand its users and share this knowledge inside their organization.

We say “Thank you, Burda” and a huge thank you as well to our partner Userlytics for the great outcome we have created with this case by working together.