Delivering ReactJS applications at scale
Introduction
Scalability of applications is an important part of application design. There are multiple ways to scale the application. The application can be scaled by writing performant and efficient code, or by adding more instances of servers. In this blog, we are going to look into scaling multiple applications built by different teams, along with robust CI/CD to support it.
Problem statement
When you have to support the development and release of multiple applications under a single entity, there are many aspects to consider i.e. scalability, release strategy, performance, security, SSO, authorization, etc. These applications have separate teams working on their product roadmap with independent release cycles. In order to solve for problems stated above, we built a hosting solution that helped the development of multiple applications. Let us look into the architecture of the solution.
Architecture
To support the development and release of multiple applications, we followed the module injection approach wherein the applications are built as an npm module, and added as a dependency in the main parent application.
As shown in the diagram above, the child applications are built by separate teams and exported as npm modules. The applications can be in any version depending upon the semantic versioning of npm.
Monorepos
If you look into the source code of many popular frameworks/library of JavaScript in GitHub, you will find that multiple packages are developed using a single GitHub repository. Below is the screenshot of the packages folder in React GitHub repository.
All of these packages are npm packages at different versions. If you have a separate GitHub repository for each of these packages, then it would be difficult to keep track of the multiple GitHub repositories. It would also be challenging to increment the packages individually when doing a bug fix or feature update. By having all these packages under a single repository, it is easier to develop and keep track of the packages.
This concept is called Monorepos. There are many such examples of monorepos in open source repositories.
Yarn supports the development of packages in the single repository by default with a feature called “workspaces”. If you are using NPM, then you can use the Lerna library for managing the packages.
Comparison with micro frontend approach
The above feels similar to micro frontend’s approach, but there are subtle differences. In micro frontend approach, the application is divided into multiple apps. Each of these apps is developed as custom elements, and usually by separate scrum teams. Below is an example of an application and its micro frontend approach.
The above example is an application about a product, divided into three separate apps — product description, product filters, and product reviews. Assume that this application is developed by three different scrum teams — A, B, and C. These teams would develop custom elements as shown in the figure below.
One of the differences between micro frontend approach and the npm approach is that we are looking at a higher level in the hierarchy of applications in npm approach. In the micro frontend approach, the application is developed in terms of multiple standalone apps with DOM as data passage layer.
In our approach, we are using the applications as npm modules. The applications can be developed as micro frontends or as monolithic. It does not affect the way we consume applications.
Similarly, the application team is free to choose any branching strategy that seems fit for their development — GitFlow or Trunk based development.
Advantages of using npm modules
The npm module approach has the framework for dependency management and versioning capability built in. The child applications can increase their version based upon the change — bug fix, feature update or breakable change. The main application adds the npm package of child applications as dependency and gets the latest version when it is updated.
Another advantage of using npm modules is that the dependencies can be shared across the applications. The common dependencies need not be downloaded again whenever the user goes from one application to another. Also, by using common dependencies, for example — all the applications use a single version of a UI library, there is uniformity in the look and feel of the applications.
Application release process
To facilitate the development of packages, we provide the boilerplate to the application teams. This boilerplate has the code quality checks built in, along with the Monorepos initialized. The development team has to clone the boilerplate and start developing their application. Once the application is developed and tested locally, developers send the Pull Request (PR) to their Git repository. This PR will run the code quality checks and test cases on the delta code that they have written. Once the PR is reviewed and merged, code quality checks and integration test suite runs on the entire code base. The application progresses to higher environments (dev, QA, stage, etc.), and finally will be built as an npm module.
This npm module is published to the internal repository of Walmart. The version of the npm module will be incremented based upon the type of change that has gone into the application i.e. patch version update for the bug fix, minor version update for the feature development and major version update for the breakable change.
Once the npm module is published to the internal repository, it will trigger the build of the main application. This ensures the latest version of the child application is available in the main app.
Continuous integration and deployment
To support the development of applications as npm packages, we required a robust CI/CD process in place. This CI/CD process checks for code quality, test coverage and also creates the npm package of the application. Following is an example of sample CI/CD process.
Here the application is at version 1.0.0. The developers start working on feature 1 and 2 in parallel. Both of these features are sent as pull requests to the dev server, wherein they get merged. Then it moves to different stages like QA, Stage, and Prod and finally built as an npm module.
Conclusion
- Centralized yet decentralized — The codebases were separate, but the applications were hosted together
- Modular code — We followed the npm based approach for the versioning and dependency management capability
- Common dependencies — Libraries were shared between the applications
- DevX — Improved developer experience and faster time to market
