Part I — Artificial Intelligence: Successfully Navigating from Experimentation to Business Value

This is the first of a three-part series on structured experimentation in Artificial Intelligence (AI).

In this article I will share a way of approaching AI experimentation that allows us to continually develop leading-edge solutions to business and customer challenges.

Customer expectations are shifting fast, especially in online retail where AI is being embedded in the user experience. Just think about how Voice Assistants like the Amazon Echo have quickly become useful to millions of consumers around the world. Likewise, take a second to consider how sophisticated Google Search has become — becoming an answer engine rather than a search engine as my colleague Giuseppe Pastore recently wrote.

We have built image recognition AI algorithms that are specifically geared towards luxury fashion, a particular sector where standard off-the-shelf Computer Vision just will not cut it

As the world’s leading online luxury retailer — it will come as no surprise that we constantly strive to provide our customers with the very best shopping experience. And like the world’s tech giants, AI is a key focus in our organisation to improve the customer experience and a core opportunity being explored by our Research & Development team’s Data Science unit.

However, the AI space is in a period of rapid evolution. Every morning I wake up and see new papers in my inbox signalling just how quickly things are moving.

This means that just building and running a Data Science team is not enough — we need to be smart about where we focus our efforts to transform experiments into truly useful products and APIs, and that can be very challenging.

Our journey with structured experimentation

Creating an environment in which your Data Science team can thrive will inevitably differ depending on the company culture, the skill set of team members and the type of work your team regularly tackles.

However, regardless of those factors, it is possible to take a uniform approach, as laid out below:

Define Your Vision

You and your team are about to go on a journey into the great unknown. It’s exciting because if done right, you’re going to be pioneers in a new field — building products and skills that did not exist before.

To get it right, we recommend:

• Setting an ambitious goal that will align and focus your team
• Ensuring that your vision is achievable with the current and future skill set of your team
• Dividing tasks by expertise and make sure you have the platforms and processes in place to ensure all learnings are shared and knowledge built across the team

In this example, we cluster different products based on visual similarity. Each line shows some results of our deep-learning algorithm applied to the NET-A-PORTER catalogue.

Create The Right Environment

To deliver fast and well, you are going to want to make sure the conditions are right for success.

First and foremost is creating space for your data scientists to focus on data science:

• Provide your team with a way to automate mundane tasks such as notebook and infrastructure provisioning, allowing them to focus on building great models, with the flexibility to choose the right frameworks and hardware combinations based on their needs
• Build a standardised way of experimentation across all projects that allows everyone to understand and contribute. It is also very important to simplify and automate the path from training to inference, in order to avoid getting stuck in a state of permanent early experimentation
• Create a single source of truth — all experiments and findings should be stored in one location which can be easily accessed by the team and shared with key stakeholders

Our detection algorithm applied to MR PORTER editorial photos

Focus on Production

It would be easy to endlessly experiment but our focus, even as a Research & Development team, is to bring new value to our customers. The only way this can work is by ensuring our experiments are totally aligned with the current and future business strategy and customer needs.

Do this by:

• Speaking to the core business stakeholders to better understand their challenges and customer experience
• Building a strong end-to-end understanding of every micro and macro process involved in their domain, specifically in the areas where there is potential to experiment
• Mapping every stakeholder involved in the potential delivery of a new function to ensure they are on board and able to contribute
• Creating test environments for the functionality to avoid disrupting the business in the early stages of experimentation
• Set deadlines and deliverables — you may be working on something new but provided you’ve followed the above steps, you should have a good idea of what is possible
• Always measure different approaches quantitatively, learn from failures and iterate quickly. This should be part of any successful innovation process
• Most importantly and it may sound obvious, but keep people engaged at every step of the journey — it is vital to have your stakeholders involved at all stages

Following this standardised approach, we have run several successful experiments which have been turned into products that are positively impacting behind the scenes processes and our customer experience.

No doubt, it is a lot of work, but you are laying the foundations for delivering AI successfully within your organisation and like building anything from scratch, that was never going to be easy 😉

In the next part of the series, I will run through the tools we use to run AI experimentation in the R&D Data Science team at YOOX NET-A-PORTER GROUP and explain why we chose them.