Your Business is Your Data
Things change fast in digital commerce. It doesn’t seem very long ago that making sure sites rendered properly cross-browser was a big deal. Since then online retailers have gone through the mobile shopping revolution, have started to take advantage of public cloud to handle scale and are now decomposing their monolithic commerce platforms into microservices.
Those are huge technological changes which have taken place over a roughly 10 year period. Most retailers are currently somewhere along that journey, some well down the road, some beginning to take steps in the right direction, and many aghast at the sheer scale of change required.
The pace of change isn’t slowing though, in fact it’s increasing. In a previous article, our CTO James Wiltshire described how digital commerce is evolving. In this article I’ll zone in on how using data and machine learning will help drive that evolution over the next 10 years. We’ll cover the following topics:
- Hype vs Reality — will machine learning really transform retail or is it all just a lot of hype?
- Machine Learning Transforms Digital Commerce — a vision of what the future looks like for digital commerce.
- Fledgling Attempts — what online retailers are doing today in this space.
- The Road Ahead — what existing online retailers need to do to remain relevant in the next decade.
Hype vs Reality
Let’s start by addressing some of the scepticism and hype surrounding AI. As is the case with most technologies, AI gets its fair share of hype. Every startup you talk to is “doing” AI in some fashion and legacy vendors have slapped AI badges all over products they built years ago to try and keep them relevant. It can be very difficult to separate what’s real, from what’s not.
The waters get muddied even further when people talk about AI but what they really mean is Machine Learning (a subset of AI). It’s understandable therefore that people switch off when listening to how AI is going to transform the next industry du jour.
The reality though is that Machine Learning is a transformative technology. It’s being used in many industries today, producing real value and its adoption is continuing to soar. Let’s defer to some of today’s industry titans for proof:
Machine learning impacts many exciting areas throughout our company. Historically, personalization has been the most well-known area, where machine learning powers our recommendation algorithms. We’re also using machine learning to help shape our catalog of movies and TV shows by learning characteristics that make content successful.
As our platform matures and Uber’s services grow, we’ve seen an explosion of ML deployments across the company. At any given time, hundreds of use cases representing thousands of models are deployed in production on the platform. Millions of predictions are made every second, and hundreds of data scientists, engineers, product managers, and researchers work on ML solutions across the company.
Airbnb Engineering and Data Science
For each search query that a guest enters on Airbnb’s search engine, our model computes the likelihood that relevant hosts will want to accommodate the guest’s request. Then, we surface likely matches more prominently in the search results.
At LinkedIn, we like to say that artificial intelligence (AI) is like oxygen — it’s present in every product that we build and in every experience on our platform.
Just think about that for a second. Every search query at Airbnb and every product at LinkedIn is driven or informed by machine learning. Millions of predictions are being made per second at Uber, and Netflix is using Machine Learning on a large scale across their organisation. Google, Microsoft, Facebook and many others all dedicate vast resources to their own AI initiatives and Amazon are obviously ahead of the curve in the retail sector.
If you find yourself trying to convince sceptics that digital commerce will undergo a similar transformation, ask them to explain what it is they know that those guys don’t!
Machine Learning Transforms Digital Commerce
The current trend towards a headless commerce system, backed by cloud-native services which are fronted by well crafted RESTful APIs is a necessary one. However, even with a highly available, scalable, well architected platform, you’re still stuck firmly in the old world of commerce. Pretty soon, that technology stack will simply be table stakes.
The next decade and beyond will be dominated by the companies who make the best possible use of their data. These retailers will be constantly experimenting and learning about what works best for each individual customer and will tailor the shopping experience for each user accordingly, across every possible touchpoint.
Data will inform them about what customers like, what they don’t like, which offers they respond to, the content that works best for each individual, which voice they respond best to and a whole host of other things, which cover the entire buying experience of the future.
Public cloud makes this type of rapid experimentation available to everyone at low cost. Retailers of tomorrow will simply take this model for granted. The next generation of digital commerce will be all about getting the most relevant products and content in front of each user at exactly the right time. In a sense, the shopping experience will mould itself around the customer. It’s sensitive to their demands and adapts as their needs change over time.
And that’s just the tip of the iceberg.
Order management, inventory forecasting, pricing, product selection and many other back office functions will similarly be optimised. You don’t have to take my word for it though. In his 2017 letter to shareholders, Jeff Bezos touched on how Amazon uses AI for those types of use case:
But much of what we do with machine learning happens beneath the surface. Machine learning drives our algorithms for demand forecasting, product search ranking, product and deals recommendations, merchandising placements, fraud detection, translations, and much more. Though less visible, much of the impact of machine learning will be of this type — quietly but meaningfully improving core operations.
Fledgling attempts
At Gradient Edge, we speak with lots of retailers about this future vision of commerce. It’s fair to say they all sit on a continuum between not using their data at all and taking baby steps into this new era. The forward thinking companies are setting up internal data science teams and running projects that may lead to business value. These projects are typically done by a standalone data science team, which looks at improving a tiny piece of the overall business puzzle.
What’s currently missing is the long term view of how data science, commerce and content are seamlessly integrated and used together, to drive real-time insights for the business and fantastic experiences for the customer. Unfortunately what we see all too often today, is short term thinking for short term gains. As James put it in his article:
In contrast, most commerce platforms today are surrounded by a plethora of SaaS services liberally sprinkled in to the technology mix, operating in silos, often with the UI layer being the point of integration via JavaScript widgets. Recommendations, reviews, analytics, personalisation, surveys, social widgets, and on, and on. All helping themselves to an organisations’ user behaviour via clickstream data, and sharing back one very thin slice of functionality in return.
Compare that with the companies I’ve mentioned above. None of them are offloading their data to third parties to get a recommendation back, or customer segmentation information for example. They realise the value in their data and are making the best use of it internally. I believe that’s the model for success long term.
Your data.
Your algorithms.
Your future.
The road ahead
A mix of sophisticated algorithms along with human creativity is required to drive these next generation digital commerce experiences. The algorithms sift through vast amounts of historical and real-time data, surfacing insights and predictions. People in all parts of the business will use these insights to guide their decisions.
Indeed, there are already companies using this model very successfully. The Stitch Fix algorithms tour gives a fascinating insight into how that company is as much a data science company as it is a retailer. As it says in that article, data science is woven into the fabric of the company.
That’s what the competition looks like for fashion retailers today and it’s a glimpse into how the next wave of online retailers will operate.
What options then do existing online retailers have to compete with these new challengers? The good news is that the incumbents typically have a wealth of data already, they’re just not making the best use of it. The first step is to simply recognise that company data is a key asset, which can be used to differentiate you from your competitors.
Once data is viewed as a key asset, the idea of offloading it to various third parties becomes much less appealing. Figuring that out should be the easy part. The next phase requires long term thinking and planning, and is as much about cultural change as technical innovation.
Data silos are a real problem for most large digital commerce organisations in the context of driving real time insights via machine learning. Algorithms require data and typically the more data you can feed them, the better. Some of the data you’ll need to pass to your algorithms in a commerce setting is, customer, order, product, price and inventory. However, that data is often stored in separate data silos:
- Customer data is mastered in its own CRM system.
- Order data is stored in an OMS.
- Product data is stored in a separate PIM system.
- Promotions, Price and Inventory may all have separate master data stores.
- Customer services systems will have their own ticketing information stored in yet another data silo.
- Content will be another system and on and on.
Bringing all of that data together to make it available to data scientists for experimentation is a real challenge. The long term solution here is to democratise access to your data across the organisation, using a mixture of data lakes, data warehousing and streaming data where each makes sense. This can be done in a phased approach but simply getting started is key.
Finally, large retailers will have to invest in their teams to make data science a core part of their business. It will be taken for granted in a few years that cross-functional teams include a data scientist, in the same way we expect those teams to have developers today. Mike Smith, president and COO of Stitch Fix explained the importance of investing in data science in a talk at Shoptalk 2019.
Investment in data science is really important to differentiate and understand your customers. If you think data engineering is a real differentiator to your business, don’t let it sit behind a group — it has a huge impact for the company and touches all functions from merchandising, planning to operations, so if you want to have a broad impact for the company, don’t bury it in a group.
Mike also stressed the importance of creating a culture of experimentation and learning :
It’s super important to have a culture of test and learn. And make sure you are constantly talking about those insights so the next test you do is just smarter.
In another article, Eric Colson, Chief Algorithms Officer at Stitch Fix also explains the importance of optimising for learning, over process efficiency:
The goal of assembly lines is execution … But the goal of data science is not to execute. Rather, the goal is to learn and develop profound new business capabilities … Perfect execution on requirements and complacency brought on by achieving process efficiencies can mask the difficult truth, that the organization is blissfully unaware of the valuable learnings they are missing out on.
The ability to go from business ideas to having actionable Machine Learning results integrated back into the experience (or optimising some back-end function) as quickly and seamlessly as possible will be essential. Doing this with data silos and a multitude of black box SaaS services just isn’t practical.
This is a big cultural shift for most organisations. Hiring the right people, fostering the right culture of experiment, test and learn, and empowering internal teams to take big risks will be the foundation that future commerce is built on.
Summary
Machine Learning isn’t a silver bullet that’s going to solve all your problems, but it is being deployed to great success at some of the largest retailers on the planet right now. It’s giving them insights into their business in a way they couldn’t have imagined just a few years ago.
As this becomes the new normal for online retail over the coming years, the organisations that succeed and thrive will be the ones that put data science at the heart of everything they do. You don’t want to be the company waking up to this reality in five years time, only to realise that you’re surrounded by competitors that look a lot like Stitch Fix. If that happens, you’re done!
