Breaking Speed Limits: Boosting Experimentation to 5x at Wallapop

This article has been collaboratively written with Bartomeu Galmés

Introduction

In 2022, our main goal was to improve user experience by offering tailored product recommendations. At first, our app lacked personalized content, so we started by manually suggesting products. But soon, we realized we needed a better, scalable solution to learn quickly and provide more value to users. Experimentation became the key to our success. This article explores how we transitioned from just three custom features to a flexible framework that now allows us to run over 15 experiments every quarter. Join us as we uncover our journey toward better user satisfaction and engagement.

Why is Experimentation important to Wallapop?

In the fast-paced world of e-commerce, staying ahead of the curve is paramount. One of the most effective strategies for achieving this is through experimentation. Some of the key aspects behind this are:

• Optimizing User Experience: Experimentation empowers Wallapop to fine-tune every aspect of the user experience, from layout to functionality. By testing various features and designs, we can pinpoint what resonates best with users, leading to higher satisfaction and increased conversion rates.
• Improving Conversion Rates: Experimentation isn’t just about aesthetics; it’s about results. By testing different calls-to-action, checkout processes, and product presentation methods, we can identify the most effective strategies for driving sales and boosting conversion rates.
• Personalization and Targeting: Today’s consumers expect personalized experiences. Experimentation enables companies to deliver just that by testing different personalization strategies based on user preferences and behaviours. This not only enhances engagement but also drives repeat business.
• Staying Competitive: In the dynamic second-hand e-commerce landscape, innovation is key to staying ahead of the competition. Experimentation allows to continuously innovate and adapt to changing market demands, ensuring they remain competitive in an ever-evolving market.
• Data-Driven Decision-Making: Experimentation provides valuable data and insights that enable informed decision-making. By analyzing experiment results, we gain deeper insights into customer behaviour and market dynamics, allowing us to learn and make data-driven decisions that fuel business growth.

In conclusion, experimentation isn’t just a strategy — it’s a necessity for any company looking to thrive in today’s competitive market. By continuously experimenting and iterating, Wallapop optimizes the user experience, improves conversion rates, stays ahead of the competition, and ultimately, drives business success.

Study Case: Recommendations in Wallapop

Problem

When we started at Wallapop, our main goal was to offer recommendations at a sustainable pace. However, learning quickly and iterating was a big challenge. We could only run three experiments per quarter due to the high effort needed to develop new features, and we didn’t have any recommendations for the app.

As we looked into the problem, we found several challenges:

1. No A/B Testing: We didn’t have an A/B testing tool in place. There was no way of segmenting our user base so we couldn’t learn about our users’ preferences effectively.
2. Limited Data Sources: Our recommendations were limited by our search engine’s capabilities.
3. Support for Multiple Components: We needed to support different UIs for each recommendation as we needed to figure out which UI worked best for each type of content and manage the order of these UIs for each user based on their behaviour.
4. Lack of Flexibility: We couldn’t change and roll out experiments quickly.
5. Repetitive Work: Retrieving items with specific characteristics for each experiment was repetitive.
6. Long Cycle Time: Our lead time was about two weeks due to a slow release process.
7. Development-Intensive: Each experiment required custom development and an app release.
8. Scaling Across the Funnel: We needed recommendations on various app screens, so the solution had to scale with minimal effort.

We believed that technology could help us overcome these challenges and build a flexible system to increase our experimentation rate. By doing so, we could learn faster and deliver value to users more quickly, reducing the time it took to complete experiments.

Solution

At first, we were developing every experiment from scratch, including custom endpoints, sliders and tracking. However, as we released more experiments we noticed that the process for generating a recommendation was the same across experiments. This led us to develop the Experimentation Framework, which provides a foundation for all our experiments, eliminating the need to rebuild common components each time and allowing us to focus on unique experiment requirements.

Initially, user segmentation was handled by the clients. This limited our flexibility, as changes to segmentation were tied to the Wallapop app release cycle. To overcome this limitation we moved user segmentation to our backend service. Shifting to a backend-driven approach allowed us to modify the UI of the recommendations without being constrained by Wallapop’s app release timelines.

Now, all sliders followed a standardized flow, pointing to the same entry point:

1. The client requests a recommendation.

2. The request reaches our servers.

3. The user is segmented using our A/B testing tool.

4. We access our data sources to generate the recommendation using its associated business logic.

At this point, we solved two of our major problems:

• Reduce cycle time decoupling from apps release train
• Introduce ab test capabilities

Despite reducing cycle time and enabling reuse for our clients, we wanted to maximize reuse in our backend. To achieve this, our talented Data Scientists developed a series of ETLs capable of processing user interactions, aggregating results using different logics required for our experiments, and outputting the results to files.

These files, containing user preferences for each experiment, are injected into our database. This allows our backend service to read them and query our data sources using the appropriate query for each user. Our A/B testing tool handles user segmentation by indicating which user data variant we should retrieve for generating the recommendation.

With reusability addressed, our focus shifted to flexibility. Client components were designed to display a variety of UIs supporting different types of entities. Our backend offers a set of parameters configurable in real-time, empowering our Product colleagues to modify experiment configurations on the fly, without the need for development or code releases.

All this enabled:

- Support for several data sources

- Changing behaviour without development

- Minimize extra work

We were at the end of our trip, but there was one more thing: We wanted our recommendations to reorder and change on key moments of the user journey, like purchasing an item.

For that, we introduced a new component in our system capable of adapting the layout of our home screen.

The key was building a rule-based system that was capable of serving predefined layouts (a set of recommendations) when certain criteria were met. Still, this was backed by our A/B testing tool as we wanted to AB test different RuleSets and layout combinations to understand which were preferred by our users.

This allows us to:

- Dynamically change the order of recommendations without requiring new releases.

- Tailor recommendations for specific user segments.

Furthermore, this comprehensive solution enables us to display recommendations at any stage of the customer journey.

Results

At Wallapop, we have systematically enhanced our experimentation capabilities over the past three years. The following graph illustrates the significant increase in experimental initiatives.

In 2022, we focused on establishing a robust experimentation framework. We conducted an average of 6 initiatives per quarter, totalling approximately 24 initiatives for the year. This foundational work set the stage for scaling our efforts for the next year.

In 2023, we introduced flexibility into our experimentation framework, allowing us to significantly increase our pace. We averaged 13 initiatives per quarter, with a total of 54 experiments for the year. The graph shows a notable increase in quarterly initiatives, reflecting our enhanced agility.

In 2024, we are applying our experimentation framework comprehensively across the conversion funnel. By Q1 2024, we made 14 initiatives, projecting a minimum of 56 initiatives for the year. This expansion ensures optimization at every stage of the user experience.

The upward trend in the number of experiments demonstrates our continuous commitment to innovation and improvement. These results (5X acceleration) highlight how a well-defined and flexible structure can drive growth and efficiency in a competitive e-commerce environment.

Lessons Learnt

Be Strategic

Strategic thinking is vital for long-term success. While addressing immediate needs is important, we must also keep an eye on future goals. Incorporating future-oriented concepts in our designs helps us prepare for upcoming challenges. This proactive approach ensures our solutions are effective today and resilient for tomorrow.

Flexibility is Key

Flexibility is essential, especially for experimentation and user experience. Incorporating A/B testing allows us to refine our offerings based on user data. Tools that control and measure what users see help us make informed decisions. This iterative process of testing and optimizing ensures continuous improvement.

Empower the Team

A key lesson learned is the value of empowering our team. Allowing team members to experiment easily fosters innovation and agility. By removing barriers and providing resources, we create an environment where creative solutions can flourish. When the team has the freedom to explore new ideas without fear of failure, innovative solutions emerge and learning opportunities abound

Tech Should Support Experimentation

Technology should enable experimentation and problem-solving. It’s essential to go beyond basic functionalities and build tech that fosters innovation. Prioritizing technologies that support rapid prototyping and iterative development helps teams test and refine their ideas efficiently. Technology should be a facilitator, not a barrier.

Keep Things Simple and Evolve

Simplicity is crucial when designing solutions. Overcomplicating processes can lead to inefficiencies. Our strategy is to build simple, scalable solutions that can evolve over time. Starting with a straightforward foundation and iterating based on feedback ensures that our solutions stay effective and relevant.

By integrating these principles into our workflow, we can create a dynamic and innovative environment that drives continuous learning and improvement. These lessons will guide our future experiments and help us adapt and improve as we move forward.

Conclusions

In this article, we explored the significant findings from our recent experiments on Wallapop. The primary focus was on the implementation of new algorithms aimed at enhancing user experience and engagement throw personalized content. Key points discussed included the methodologies employed, the statistical outcomes, and the practical implications of these results.

The results of our experiments have already started to influence daily operations at Wallapop. By integrating the new algorithms, we have observed an increase in user retention and satisfaction. Looking ahead, these findings open several avenues for further research, including refining these algorithms for even more personalized user experiences and exploring their applications in other areas of our platform.

We encourage readers to apply the insights shared in this article to their own projects and experiments. If you found this article useful, please share it with your colleagues and networks. Additionally, we invite you to leave your thoughts and experiences in the comments section below. Your feedback is invaluable in shaping future research and improvements.

Thank you for being a part of this journey towards innovation and excellence.