Embracing an experiment-oriented mindset at ASOS
How experiments have helped ASOS stay customer-obsessed and positioned us for success.
You might not know it, but when shopping on ASOS, you likely experienced one of the hundreds of experiments we run.
I want to talk about what experimentation means for us and how it enables us to live up to our core values; Authentic, Brave, Creative and Disciplined.
What is an experiment?
You might think of an experiment as being in a science laboratory, two flasks sitting on a counter with green and purple bubbling liquid, and a scientist with a clipboard writing lots of things down.
It’s kind of like that.
You might have heard the phrases A/B test or MVT (multivariate test) used.
In the context of the ASOS app, imagine something specific—like changing the colour of a button from green to blue.
Without experimentation: Everyone gets the blue button.
With experimentation (hypothetical scenario):
- Hypothesise that a blue ‘Add to Bag’ button will encourage more users to add a product to their bag, when compared to a green one.
- Decide to split the users in half. 50% of users will see a green ‘Add to Bag’ button and the rest see blue.
- Observe and compare how many users add a product to their bag, along with any other metrics, for a given amount of time.
- Once data analysis has finished, roll out the colour which had the better performing metrics, to the entire user base.
What do experiments mean for ASOS?
Being the top destination for fashion-loving twenty somethings, we need to stay on top of trends and reduce as many barriers to innovation as possible.
Experimentation lets us try new things.
Making a change to the homepage or a product page, both high-visibility parts of the customer journey, comes with a risk as customers may react adversely to the changes.
The ability to allocate a specific percentage of customers to include in the experiment, and to drive the user experience using remote settings, allows us to test changes within a more controlled environment.
Being able to pilot new features and functionality in a reliable and safe environment gives us more confidence to continue innovating.
Experimentation empowers our customers.
At ASOS, we’re customer-obsessed. We strive to always be in tune with our users and ensure a seamless customer journey.
We also value getting new improvements and features in the hands of users as effectively as possible. A key part in making that possible is reducing unnecessary feedback loops and avoiding biases.
Consider a scenario where the design team have a few excellent concepts for a particular UI component, but can’t decide which one the customer would prefer.
A traditional approach would be to use a focus group. This isn’t a bad way in principle to gather initial feedback — however, there are some difficult questions:
- How large should the focus group be?
- Is the group representative of the customer base?
- Will every member of the group be available?
- Do the group members have any biases?
Is using a focus group the optimal way to deliver a feature in a timely, optimal fashion?
Instead of a handful of people in a focus group, we can leverage the millions of active users that shop using the ASOS website and apps.
With experimentation, you can test the different variations of the UI component across the entire customer base. The customers can decide which variation they prefer, and we can determine that based on what metrics and data we collect.
Experimentation makes us more agile.
When you embrace experimentation, you embrace iterative design and incentivise a more agile mindset at a product level.
Shipping a feature after months of conceptualising, developing, back and forth with business stakeholders, carries an inherent risk of wasted time if customers reacted adversely.
With experimentation, we can work against a roadmap, making small iterations and testing each change.
The above approach suits agile software development as we work in 1–2 week release cycles, which makes things run just that little bit smoother.
Why risk a leap of faith, when you can take measured, calculated steps and know you’ll end up in the right place?
Experimentation holds us accountable.
Being Disciplined is also one of the core values at ASOS.
As discussed above, the beauty of experimentation allows us to compare the reaction between customers on an iterative change in the product — tracking against business KPIs and other metrics.
The product team has more insight than ever into the impact of their portfolio. This transparency into performance empowers us to hold ourselves accountable and celebrate successes and innovations.
Experimentation helps us learn.
If there’s anything you want to take away from this article, it’ll be this.
Experimentation instils a test-and-learn mindset, applied not exclusively to the colour of a button but how the business approaches different processes throughout the product development lifecycle.
It helps us evaluate and optimise the processes we have in place and how we approach product roadmaps.
We can develop features quicker and make more informed, timely decisions — bringing us closer to the customer in the process.
Sometimes we make mistakes, but that’s ok.
Conclusion
Experimentation throughout ASOS keeps the customer at the centre of what we do, gives us greater insight into the impacts of our work and holds us accountable to our core values.
About me
Hi, I’m Callum. I’m a Senior iOS Engineer @ ASOS. If you didn’t notice, I love experimentation and writing about it. Thanks for taking the time to read my article. You can find more pieces by me on the ASOS Tech Blog and on my profile.
ASOS is hiring across a range of roles, see our open positions here. We’d love to hear from you!
