User Research with the Jumbo Online Test Team: A Unique Approach

In the realm of Design and Product Management, creating products and services that meet customer needs is essential. To achieve this, understanding those needs is a crucial aspect of the design process.

However, conducting customer research can sometimes be quite a hassle. Common hurdles include:

• Difficulty in consistently reaching the target audience
• Inadequate responses to research studies
• A cumbersome process for engaging with customers
• High costs associated with external research panels or agencies.

Inspired by a suggestion from Ilse Blom, a freelancer who used to work in my team who highlighted the uniqueness of our approach, I decided to document how we established our own customer panel: Jumbo’s Online Test Team.

Better and More Cost-effective

Several years ago, Jumbo initiated the creation of the Jumbo Online Test Team. This customer panel managed by ourselves consists of a group of representative Jumbo customers whose behaviour and characteristics align with the customers we aim to serve.

The panel’s objective is to give customers a voice in shaping the ongoing development of our products and services. We’ve organised this in such a way that it, theoretically, allows us to move from a research question to customer insights within just one day! Pretty cool, right?

Key Facts

• The test team currently contains around 12,000+ active members.
• At least one research project is sent to the panel every week.
• Common research types include surveys, click tests, and interviews and usability tests.
• The panel focuses on product development, improvements to our propositions and customer journeys

6 Key Elements

In order to explain to you how the test team works in practice, I’ve highlighted several key elements:

1. Exclusive Invitations

We want the panel to reflect our primary target audience. Potential members receive an exclusive invitation via email if they meet specific criteria, such as having placed more than 5 orders. This happens through an automated email campaign. Upon accepting the invitations, an opt-in is automatically added to our customer database, allowing us to approach these customers for research.

2. Conducting Research

Customers receive a approximately one research project per month to prevent communication overload. This happens via email. Upon receiving a research project, the customer reads a brief summary, learns about the research type and estimated duration of the research, and, by clicking a button in the email, starts the study.

3. Sending Research

Most of our service designers, CRO specialists and UX designers have the autonomy to compose emails, select a segment, set up the research, and independently send it. This way, we are not dependent on other colleagues or external agencies. The Customer Journey team will keep an eye on quality assurance of the studies.

4. No Compensation

We’ve chosen not to provide financial compensation for participating in smaller surveys. For larger research studies, rewards are offered. However, when customers sign up for interviews or usability tests, we don’t mention rewards in the invitation, aiming to maintain intrinsic motivation.

5. Expressing Gratitude

To maintain a strong connection with our panel, we send customers an update every quarter. These updates include key findings from recent research, updates on changes to our services based on their feedback, and a preview of what we expect to develop or research in the coming months. And besides the quarterly updates, during Christmas we also express our gratitude to our most active members by sending them a small gift and a personal Christmas card.

6. Unsubscribing and Inactivity

Customers can always unsubscribe via their Jumbo account. Unfortunately, we observe instances of inactivity, so we conduct a yearly cleanup, reaching out to inactive members via email to confirm their continued interest. If there’s no response, we automatically remove them, ensuring the active status of our test team.

What it has brought us

Test team members have contributed to hundreds of research projects, helping us determine what truly matters. The Jumbo Online Test Team has evolved into an important group in our design- and product development process, ensuring that we maintain a strong product-market fit with our customers.

Collaborating with the Jumbo Online Test Team has led to numerous achievements, from UX improvements to proposition and branding upgrades. Some examples include:

• UX optimizations: Let the test team decide what information should be displayed on a product card in list views.
• Branding: Collaborating on the name and logo of our subscription service.
• Touch points: Identifying preferred channels for receiving order tracking information.

This is how we give customers a voice into our decision making process.

A big shout out to Sander Freijzer who was responsible for setting up the first big steps towards Jumbo’s Online Test Team