Squaded: TDD and Microservice Architecture for widget

Squaded is an embedded widget that works on E-Commerce sites of clothing and accessories, such as Zara and Asos, allowing you to communicate in the community, compare prices and turn the boring process of searching and selecting the right product into a fun conversation and discussion of photos with your friends.

Subscriptions, tracking subscribers, likes, the ability to create your private community, post your photo from the fitting room in a particular outfit and start voting — this is only part of the overall functionality of the widget. At the same time, each of the described functions should be associated with products of the site where the widget has been integrated.

The seller, in turn, receives a huge array of data for analysis — which products are most in-demand and popular at the moment. This allows them to subtly feel the needs of customers and quickly respond to them, thereby gaining an advantage over competitors.

The project implements a microservice architecture consisting of 6 microservices, each of which is responsible for certain tasks in the application. This architecture has been chosen because of its advantages:

• ability to easily and quickly scale high-load system services if necessary, and isolation;
• the ability to create new features without affecting the work of existing services, and interacting with each other using network communication protocols, which allows you to quickly add new features, responding to user requests and the customer.

As for the technologies used in the project, the choice fell on TypeScript + Vue.js in the frontend, TypeScript + Node.js in the backend part, MongoDB as the database, which allows you to easily and quickly scale the project when the load increases. All this was deployed in AWS EC2, and many of its services were used, such as S3 for storing user photos, SES for mailing lists, Rekognition for filtering unwanted content, and so on. As for the alternative to MongoDB from AWS-DocumentDB, it did not work because of the lack of support for uncorrelated queries (multiple join conditions and uncorrelated subquery) about tasks.

To increase the efficiency of the high load, we decided to use the Mongo API directly because the ORM of a system like Mangoost cannot provide the required level for a large number of JOIN (lookup) requests. One of the most difficult tasks was creating and profiling very complex queries involving multiple tables (collections) (up to 7).

To achieve a good query execution speed, I used pipeline processing — Mongo Aggregate — some of the pipelines had about 20 stages. Writing code for these functions and then debugging and profiling to work more efficiently has become one of the most difficult tasks on the project and, of course, required the use of special tools and services — such as Studio 3T for writing and debugging aggregate queries; and AWS CloudWatch and Metrics — for logging and evaluating the speed of queries, ultimately ensuring fast processing of request queues, and high responsiveness of the system to user actions.

Due to the constant addition of new features when it is necessary to maintain 24-hour project performance, one of the customer’s requirements was to cover the entire Unit code with tests following the TDD concept. To do this, we had to create quite complex mock functions to emulate interaction with MongoDB, Redis, and other microservices.

The result of the project was an adaptive, fully functioning, embedded widget for e-Commerce sites and, for me personally, knowledge in microservice architecture, as well as increased skills in writing Unit tests and working with databases at a low level.