Our journey to Micro Frontends: Part 1
This blog post is going to dig into why we’ve decided to migrate our monolithic frontend project into a micro frontend architecture. This post will be the first part of a series where we cover why and how we’ve implemented this. We’ll take a look at the technical details, as well as some of the problems we’ve encountered and how we’ve solved them.
Our main product at Peak, CODI, is a very large, complex, distributed application, spanning hundreds of git repositories. We’re heavy users of microservices and use a number of AWS services to reliably link these together. However, in the case of the frontend UI, this is very much a different story. The frontend is a growing monolithic React application, with over 140k LOC, and (at the time of writing) around 35 engineers working on the same product.
With the company recently closing Series B funding and rapidly expanding with a significant hiring plan for the future years, we knew we were going to start facing problems with our current codebase when these numbers rise and the application inevitably becomes more feature complete.
What problems are we facing?
- Developer experience is a big problem. The application is slow to start up and consumes large amounts of memory. On older laptops and less powerful ones we would often see initial build times taking 5–6 minutes
- Deployment of the frontend is slow. CodePipeline takes about 5–6 minutes to install the dependencies, build the code and upload to S3
- Development builds running in the browser use a lot of memory, and don’t feel as snappy as they would when running production builds
- Having to spin up brand new environments to test code while other code was going through QA. Although most teams have their own environments alongside the standard dev, and test ones, we occasionally meet bottlenecks when only one thing can be deployed at any given time, without spinning up a new environment
What is the ideal solution?
It was clear we needed to split the application up, somehow. However, the ideal solution had some important requirements for us.
- Some separation of concerns in an ever growing complex application
- Fast build and rebuild times
- Fully integrated SPA (Single Page App) experience, no refresh required when navigating between areas in the app
- Independent deployments so they’re quick and risk is minimised
- A better mechanism to test different branches without spinning up entirely new environments
- Minimised lock in to tooling (we should be able to change tools without having to replicate many changes in all of the projects)
When we started thinking about this in 2020, there were a number of different paths to take and we considered all possibilities. From the solutions we initially looked at, though, they didn’t really solve all of our problems — and this left us looking for more. I’ll cover some of the possibilities we considered.
Entirely separate frontends for different solutions
Separating the frontend out into separate websites kind of felt like a backwards step, leaving our fast SPA behind and forcing users to refresh between each area within the app. There was also complexity involved in keeping the applications up to date with shared components (like the header, and navigation menus). Ultimately it would have left our users with a reduced experience and a website that performs worse due to duplicated npm modules. This was unacceptable for us.
Separate npm packages for different solutions
This solution probably feels more similar when working on a microservice style architecture, allowing us to split the application up and still achieve a consistent SPA experience. But ultimately it still needs to be combined at the end when building — which would be faster, however it would still take us back to a centralised deployment mechanism which we wanted to avoid.
This also introduces complexity with updates and keeping all the micro frontends updated. It felt like the “container” application was going to become a bottleneck. There was also another problem here, when working on the individual projects there would be no surrounding application, no header, no navigation — although I’m sure this could have been solved by running the container with the ability to point modules to local packages. This actually wasn’t too bad of a solution and works pretty well in a monorepo style architecture with something like lerna (we’ll discuss this more in the next blog post.)
Module Federation with Webpack 5
It was by chance that we noticed Webpack 5 was being worked on with Module Federation when we started planning this project. This ticked all the boxes, and there was lots of community hype surrounding this, such as blog posts, YouTube videos, etc. This is exactly what we’ve been looking for; entirely separate builds and deployments, a unified SPA experience, shared packages, and — importantly — a full website experience when running the individual solutions.
What is Module Federation?
So, what is Module Federation? I’m not going to go into a huge amount of detail in here, as it’s been covered extensively by many. But in a nutshell it allows you to combine separate webpack builds together at the runtime, effectively mimicking the behaviour of npm modules which are fetched over http when your users visit your website.
You might think, “well I can do this by simply including individual bundles on my page,” and you’d be right. However, the important difference is that those bundles don’t behave as one in the above example — when you connect them with Module Federation they behave as one app, and can share dependencies and everything else. For example, you can have one Redux store and have your individual federated modules access and enhance that Redux store.
Are micro frontends the right solution?
It’s fair to say that there has been quite a lot of discussion on this topic. Some people agree with it, some people don’t. There is an argument for having your codebase split up into separate component libraries, and an argument for having separate deployments entirely, and to be honest, there is a use case for both as well, in my opinion.
Perhaps it comes down to an organisational problem, where team autonomy and isolated deployments are more valuable than isolated component libraries. I think that’s the case for us here at Peak anyway. But, I can confidently say that I’ve never worked on an enterprise frontend project that hasn’t eventually needed to be split up from a monolith into something else, for various reasons.
First steps
Around mid 2020 we took our first steps and blocked out two weeks to do a proof of concept. Two of us split up the basic tasks we had to demonstrate a potential working solution. We split the work up into two parts, moving the authentication and “container” code (by this I mean navigation bars and menus) into a new repository, and the tooling/infrastructure work. We initially decided to use Webpack 5 in beta so we could make use of Module Federation early on while it was still being tested.
The proof of concept was a success, and we demonstrated how we can split up the application into two parts, the container code and the “legacy” structure — which basically mounted the monolithic application without the menus. The solution was very basic and had a lot of rough edges, but the main thing that we took away was that it was possible to deliver this solution without any detrimental effects on the developer experience. And what I mean by that is, while developing in the legacy application, we were able to mount the container code and provide a full E2E application experience (login and menu), without running the container as well.
There is another part of the developer experience that was essential for the success of this project, which isn’t something that you would usually think about when building frontend applications, and that’s consistent tooling. Tools like Create React App, Next JS, Gatsby — I think a part of the reason these tools are so popular now is because of how easy it is to get from nothing, to a working project where you can immediately start writing business logic. Anyone reading this will probably be aware of the myriad of tools available, and even required for getting a basic SPA running these days. Webpack, Babel, Jest, ESlint, Prettier, TypeScript, various Webpack loaders, the list can go on and on, and to be honest anyone who has set all this up probably knows it can be a nightmare to maintain.
Now, traditionally you’d maybe configure this all for your website and then when you build a new website, you’d maybe copy it, or you’d just use an off-the-shelf tool that does all of this for you. We knew this was a problem that we had to solve, we couldn’t just configure all of this and have it duplicated every time someone creates a new micro frontend. How would we roll out updates, how would we keep things synchronised, and how would any of this code be maintainable?
It was fairly clear we needed a toolset, but one that ticked the following boxes:
- Centralised configuration
- Easy to update
- One (or few) packages to install instead of many
- Quick to get started
As part of the proof of concept we created a local npm package which installed Create React App and used React App Rewire to allow users of the package to customise certain things, and configure Module Federation. This actually only turned out to be a small file, but it demonstrated that we can achieve all these goals.
Next steps
In the next post in this series we’ll cover how we took this proof of concept and turned it into a deliverable project that met all of our requirements, some of the difficult problems we encountered, some other cool things we’re pretty happy with, and what we’ve got planned for the future.
