How to leverage quantitative research to drive faster design outcomes
Principal Product Designer, Joydeep Sengupta, and Senior User Researcher, Dan Balica, share learnings from their mission to turn around a high-priority brief within just three weeks.
How much do you trust a good discount? The psychology is clear: greater price reductions make for easier purchase decisions. Still, for today’s savvy shoppers, some discounts just look too good to be true. There are two main explanations: either the price of the product has fluctuated over time, or the vendor has deliberately hiked up the price ahead of the sale to display a greater discount. The latter is known as price tricking, and it’s something the EU Council’s Omnibus Directive, effective since May 28th, 2022, is cracking down on.
The new regulations aim to safeguard consumers’ rights by increasing the transparency of the offer. Stores are now obliged to provide the lowest price at which the product was sold, for at least the past 30 days, so that customers can judge for themselves how good of a deal they are actually getting.
Project Omnibus
At Zalando, we are always looking for ways to strengthen our relationship with the customer by cultivating trust and increasing the value we offer. When the brief to redesign our product cards to reflect the new pricing regulations landed on Principal Product Designer Joydeep’s desk, he wasn’t short on enthusiasm. Due to the high priority of the assignment, he wasn’t short on resources either. However, he was very short on time.
With roughly three weeks to complete Project Omnibus, Joydeep joined forces with Senior Quantitative User Researcher, Dan Balica. He is part of our Voice of Customers team, which provides user research and customer insights for teams across Zalando. To ensure compliance, Joydeep also worked in close collaboration with Tanja Praast and Fabian Piltz from the legal and commerce teams, and Product Managers, Florian Graf and Aniruddh Ojha.
Joydeep and Dan recently shared their learnings with our wider product design community in a session presented by Senior User Researcher, Shaun Sandu. As a recent joiner to Zalando, specializing in qualitative research, Project Omnibus was one of Shaun’s first points of reference to learn more about the mature quantitative practice that sparked his curiosity. “I was intrigued by how Joydeep and Dan were able to work creatively to leverage lean, yet rigorous, quantitative research to decide between several different designs in a matter of weeks, rather than months, as was my initial expectation,” Shaun reflects. “It was also a good insight into how to balance user experience with regulatory constraints.”
Are you interested in finding out how Joydeep and Dan achieved their goal? Read on for their insight into the challenging project.
Assumptions at the outset
Since the directive had not yet come into effect when Joydeep started the design process, he did not have any reference points from competitors. From the legal perspective, displaying all three of the prices — the original price, the price from the past 30 days, and the current price — was the most watertight solution. However, from the design and commercial perspectives, the assumption was that this was likely to negatively affect the customer experience and the business performance. More information on the product card would not only increase the cognitive load for the customer, but it would require a larger product card size, reducing the number of items in the viewport.
In order for Joydeep to translate the legal directive and commercial requirements into a positive customer experience, he had to test the assumption: “I decided to put it in front of the users to see if our intuition was correct.”
Deciding on the methodology
That’s where Dan stepped in with his research expertise. As the differences between the designs were quite subtle, he decided a quantitative method would generate the most meaningful results. The time constraint also factored in against conducting qualitative user tests. “We had six designs, and if we were to include at least five users, then we would have ended up with 30 to 60 hours of interviews,” says Dan. “We just didn’t have that much time. We decided to go with a ‘between-subject’ survey, where each tester experienced only one design, then answered a set of benchmark questions. This method would allow us to use the same metrics and KPIs to track each of the designs.”
Collaboration challenges
As a Principal Product Designer, Joydeep is especially switched on to the business impact of design solutions: “Our responsibility as product designers is not only to create delightful customer experiences, but to make sure that we are using our time and resources efficiently.” As such, Joydeep and Dan set a deadline by which to send out the survey. Between receiving the brief and sending out the survey, Joydeep created hundreds of design iterations. The biggest challenge of the pair’s collaboration was that, due to the uncertain nature of the project, Joydeep was incorporating feedback from legal and commerce right up until the last minute. To reflect the changes, Dan had to keep readjusting the survey.
“We started out with a tight timeline of two or three weeks, and we ended up delivering the solution in about four weeks,” Dan reveals. “We had to be pragmatic and prioritize which hypotheses we needed to answer at this stage of the journey. We reached a consensus, we had a plan, and we successfully overcame this challenge.”
Research findings
Was there anything surprising about the survey results? The user researchers out there will certainly relate to Dan when he says that he loves boring results! “I get worried when I get unexpected findings,” he laughs, “because then the question is, what went wrong? Were our assumptions off, or more likely, is our data off? Was there something wrong with the data and the way we collected or quantified it? I want results that make sense, and this was the case with this project.
We didn’t have anything that was particularly striking or that shocked or surprised us.” The only unexpected result was the magnitude at which the simplest of the designs underperformed. “The design without a red discount percentage, crossed-through original price, or recommended retail price, performed by far the worst compared to the rest, which indicates that sometimes less is not more, and less is simply too little.”
Utilizing the findings
The quantitative method may have been the least time-consuming research option available, but that didn’t mean it wasn’t thorough. Leveraging our 16,000-strong customer panel, the Voice of Customers team managed to collect over 2,500 responses to the survey. This gave Joydeep a solid position to present to his legal and commerce stakeholders: “Now we were not talking about hypotheses, but the options that thousands of people said they wanted to see. We presented them on a scale of lower to higher commercial and legal risk. The constant of all of these options was the customer experience.”
Utilizing specially created Zalando Design System (ZDS) components, the final design incorporates the initial prices within an extra line of the product card, which is only visible when there is an Omnibus price. Time will tell whether it will pass EU scrutiny with flying colors, and how the changes will affect Zalando’s bottom line.
Key takeaways
What did Dan and Joydeep learn from their collaboration that could help others embarking on similar projects? Dan’s major takeaway was that as long as everyone involved in a project is fully committed, and the resources are in place, almost anything is possible. “I was in constant communication with Joydeep,” he says. “When I asked him for feedback, I’d get it the same day or the next day; there was no loss of momentum, and therefore, no loss of time. We delivered a high-impact project in a reasonable amount of time; we accomplished a lot.”
In part, Joydeep owed his timely manner to the fact that, due to the high priority of the project, he was able to dedicate himself to it fully and put everything else on the backburner. “I felt like I was flying Air Force One and didn’t have to wait on the runway,” he laughs. According to Joydeep, the project was also a great example of one of Zalando’s founding principles, ‘Live High Challenge and High Support,’ which treats feedback as a gift and support as a form of care that exceeds the limits of professionalism.
Has your design team leveraged quantitative research to turn around a high-priority project? Do you have any advice for designer-researcher collaborations? Let us know your thoughts in the comments.
