From Legacy to Serverless: A Returns Novel
Last year, SSENSE formed a new squad to focus on customer returns. One of the main goals of this newly formed team was to migrate the returns system, that resided in the legacy codebase, to a brand new serverless microservice based on AWS technology. I consider myself fortunate to have joined this new team as I am getting firsthand experience designing and building a new microservice from scratch and seeing it replace the legacy codebase that was in use for several years.
When SSENSE operated on a smaller scale, code from different teams resided in the legacy codebase. The code was often coupled, making it difficult for developers to add new functionality as a bug introduced by one squad could risk breaking the entire system. As a rapidly growing company, teams at SSENSE are peeling off their code from the legacy service and adopting a microservice-based architecture. For the returns squad, it was imperative to build our own microservice to provide greater flexibility in supporting the expansion of SSENSE Fulfillment Centers.
In this article, I will explore the solution design our team created in order to migrate from a legacy monolith to a serverless microservice. The transition can be broken down into four steps:
- Synchronization
- Taking control
- Becoming the source of truth
- Removing legacy
Step 1: Synchronization
As a first step, our goal will be to have the new returns microservice and legacy system fully synchronized with each other. We will not make any changes to the legacy codebase during this stage.
Now let’s explore how we achieved the goal of synchronization between the legacy and new systems.
The SSENSE tech ecosystem utilizes event-based communication. Whenever a service creates or updates a return, it emits a notification containing information about the return. These notifications are emitted at each step of the returns lifecycle. The lifecycle of a return at SSENSE can be broken down as follows:
Lifecycle of a Return
- The customer requests to return items they have purchased.
- SSENSE generates a return label for the customer.
- The SSENSE warehouse receives the merchandise returned by the customer.
- The customer gets refunded for the items returned.
The team will build a brand new microservice from scratch and the entry point to this new returns system will be a queue that subscribes to the return messages emitted in legacy after each step of the returns lifecycle. Whenever our queue receives a message, we will pass it down to an AWS step function. Based on the topic of the incoming message, the first step function will pass the message to a lower level step function that either creates or updates the return. When we don’t have all the required information, the initial step function will invoke a lambda function to fetch the missing data from legacy before passing it down to the lower level step function.
The lower level step-functions will orchestrate a series of lambda functions that validate the incoming message, format the message based on our internal schemas, and save or update the formatted data into our database. Finally, our system will compose and publish our own messages so that other domains dependent on returns can consume them in the future.
After this implementation, we have both returns and legacy systems running in parallel. All new returns will now enter the new returns microservice and be persisted in its database. However, we still need to import all the historical returns data to achieve 100% synchronization with the legacy system and ensure backwards compatibility. To do this, we wrote a script to export the historical data from the legacy database, format the data according to our data model, and persist the data in our new returns data store.
Step 2: Taking Control
Now that we have created a brand new serverless microservice, we want this service to take control of returns. The legacy system will still be doing all of the heavy lifting at this point — creating and processing returns and providing returns data to other services through emitting messages or exposing endpoints.
As we want our new service to eventually replace the legacy system, in step 2 of the peel-off, we will take control of creating and updating returns. This can be achieved using an AWS API Gateway that exposes endpoints to create or modify returns, and to retrieve returns data. The API gateway communicates with the step functions and lambda functions implemented in step 1 to retrieve and save data. Once the API Gateway is set up, services using the legacy endpoints for returns must transition to calling the endpoints exposed by our new system. To move away from dependency on the old system we will also remove the lambda function that fetches returns data from legacy.
Even though we control returns at this stage, we still want to maintain the synchronicity with legacy. This is achieved using a queue that subscribes to the new messages emitted by the returns system. A lambda function consumes the new returns messages from the queue, and uses the endpoints provided by legacy to post data to the old system.
Step 3: Source of Truth
In the third step, we want to become the source of truth for returns. Domains dependent on returns information still rely on legacy messages to obtain information, but returns are initiated and processed by our system and we are emitting messages with all the information about a return whenever a return is created or modified. So, the foundation for a transition is in place.
In this step, we will identify all the domains that retrieve returns information from legacy and communicate with the respective teams to facilitate the necessary changes so that they consume the messages emitted by the new returns service.
Step 4: Removing Legacy
In the final step of the peel-off, the new system has taken control over all the returns processes through emitting messages to SNS topics and the API Gateway. Other domains that previously relied on legacy notifications are now subscribed to our new system’s notifications.
Now that we have no dependency whatsoever on legacy, we can finally remove the returns code from the legacy system. We will place any returns logic that lived in the legacy system under feature flags. After thoroughly testing the entire flow, with feature flags off to ensure the old code not being called doesn’t break any existing functionalities, we will gradually remove the code from legacy.
Final Thoughts
It has been a wonderful learning experience working alongside software architects and developers in my team throughout the process of designing the peel-off. I had the opportunity to work on the architecture design studying old code on the legacy system, identifying the dependencies returns have with other domains, coming up with the data model for returns, and finally implementing the new serverless system from scratch.
We have completed the design for the architecture peel-off and are partially through the process of implementing it. Moving forward, my team will have greater flexibility in adding new functionalities and modifying the returns system.
Editorial reviews by Catherine Heim & Nicole Tempas.
Want to work with us? Click here to see all open positions at SSENSE!
