Powering the Supply Chain with Serverless Architecture — Part 1
This is the first part of an article about how we use server-less architecture at Gousto to power our supply chain operations. This part elaborates on our journey to the new architecture, and the second part details the architecture itself along with benefits and drawbacks.
Introduction
At Gousto, we pick, pack, and deliver millions of meals to our customers every month.
The process of fulfilling these orders is enabled through a complex set of supply chain operations. These are managed through our bespoke e-commerce platform on AWS, combined with industry-standard Enterprise Resource Planning (ERP) software and a dedicated Warehouse Management System (WMS).
Our customers can choose from a weekly menu of over 30 recipes in a variety of portion sizes, which means our systems can potentially process over 200,000 unique combinations of ingredients, and millions of individual order lines in any month.
Gousto featured on the 2018 Sunday Times Tech Track 100, as one of the fastest growing private Tech companies in the UK
The extraordinary growth Gousto sees every month, combined with an exciting roadmap of proposition enhancements to provide flexibility and choice to our customers, adds significant complexity to our supply chain operations. All of which need to be managed, analysed and improved continuously to ensure that we deliver orders on-time and in-full to all our customers.
Genesis and growth
The first generation of software to manage Gousto’s supply chain was introduced in 2014.
It consisted of a traditional ERP system from a small UK-based vendor, a dedicated WMS to manage our complex warehouse operations, and a set of cron jobs for synchronising transactions between the two systems. Interfacing with the software occurred using Windows desktop clients, and through batched CSV files on file servers.
This architecture met all our requirements for managing stock, purchasing ingredients in bulk, producing shippable packages in precise quantities, and packing the correct ingredients in boxes, based on volume and predicted growth at the time.
The emergence of an in-house engineering and data science team led to the development of a custom e-commerce platform on AWS. Combined with the automation of our warehouse in Spalding, business growth exceeded all expectations over the next few years.
The ERP system — the backbone of any retail business — started to creak under the weight of this growth, and in late-2016 we decided to begin the process of replacing it.
Build vs. buy
Knowing that this was a critical juncture in our journey from start-up to scale-up, we needed an ERP solution that could support our growth for the next five years and beyond, both in terms of volume and proposition enhancements.
This presented an opportunity to design part of our supply chain operations architecture from scratch, leveraging our in-house engineering capability and the platform we had built on AWS. This was also an opportunity to address a number of limitations with our first-generation ERP system and integrations, and provide additional value to the business instead of an “as-is” replacement.
- Recipes and stock were being mastered in multiple systems, with independent ownership, which was causing discrepancies and integration failures between the systems.
- Sales orders from our e-commerce platform underwent manual supplementations through our ERP system and WMS, which meant the time between order confirmation and picking was measured in hours instead of minutes or seconds.
- All data transitioning between the ERP system and WMS required batching into CSV files, which meant failure due to a single data entity caused the entire batch to fail.
- Business users had minimal insight into data transactions between systems, and detecting failures was often a case of missing expected data.
Whilst we wanted to build a state-of-the-art system using modern technology following modern development practices, we knew that building a full-fledged ERP system ourselves would be a formidable challenge, and not the best use of our engineering resources.
After multiple rounds of deliberation on “build vs. buy”, we decomposed our desired architecture into a set of logical components, based on mastering of entities, functionality that could give us a competitive advantage, and what we saw as undifferentiated heavy-lifting in an industry-standard ERP product.
This decomposition allowed us to evaluate what software we could develop and iterate on ourselves, and software we would need to buy in the form of a new ERP system.
Guiding principles
With our logical architecture agreed, we established principles around which we could design the physical architecture and integrations between the Gousto platform, the WMS, and any new ERP system.
- Transactions should be driven by events, whether in our platform, the WMS, or the new ERP system.
- Each event, and therefore transaction, should be processed independently of all other events.
- Transactions should be platform-agnostic in transit, and should be transformed into platform-specific entities only at system interfaces.
- Integrations should be stateless and fault-tolerant, with automatic error- and retry-handling.
- Integrations should provide sensible metrics about data flow.
- All software and integrations should scale on-demand in a cost-efficient way.
These principles, along with the decisions around our logical architecture, fed into the technical requirements of the vendor selection process for our new ERP system. After a thorough selection process, we settled on a SaaS ERP product from an industry-leading vendor.
Follow the Gousto TechBrunch blog to read the second part, detailing the new architecture.
