Scaling transformations through simplicity
By Ankush Mahajan, Principal Engineer at Macquarie Group
Overview
As we continue our journey of simplifying transformation across Macquarie, a key focus of my role is ensuring architecture and engineering not just enable the simplicity of designing and building solutions, but also how our people can have autonomy to build solutions through the tech and tools they know best. Over the last few years, ‘digital transformation’ has been a focus in many major organisations, and I’ve been very fortunate to have worked on several of these kinds of programs. Across all of these transformations, I always ensure the outcome reflects the 3S approach: Simple, Secure and Scalable.
In early 2020, we were tasked with designing a seamless, consistent and unified world-class experience for our employees, enabling them to access multiple internal services in one place. In this article, I share the approach and high-level design that not only made the transformation successful but paved the way for us to think beyond this domain and apply the same principles for other transformation projects.
The challenge
Our challenge was to build a ‘one stop shop’ that brings together all the different staff-centric services, such as leave and payslip services, to give users a consistent, unified experience at their fingertips.
The key was for these teams to have the autonomy — within approved architecture — to choose the best tool to use to deliver the business outcome. This requires cross-functional team collaboration in order to integrate and deploy continuously, enabling scalability as well as a fast and efficient roll-out of services.
Microservices, of course
Microservices, if done right, help engineers draw out the right domains and find solutions for those domains (if you’re solving with the business and not in isolation). Microservices allowed us to deploy these domain solutions in isolation to each other as well. Once we had solved for the bounded context we were mostly set. There will always be challenges in service-to-service communication, visibility, observability and keeping things in sync but there are well documented patterns, tech, and tools to solve these. There’s much more complexity to this but for this article we’ll focus on the micro frontend approach we took. As a community we have come a long way in designing the microservices well, however we were still missing something.
Let’s recap the challenges:
Business challenge — One-stop shop to bring together different offerings.
Technical challenge — enabling nimble cross functional teams to have the autonomy to choose from the different tools and integrate and deploy into the product continuously.
We would have fallen short at both — even though we had well-defined microservices, our front end was one huge monolith and the complexity of it remained a challenge.
Rise of the Micro-frontends
When we started, we had either a single frontend team or a range of different teams contributing to the monorepo frontend. In this environment, teams were not able to choose their own technologies and tools or deliver and deploy independently. Teams needed to cherry pick commits from lower environments, and branches became more complicated as the team grew.
So, we decided to break down the front end into multiple micro frontends. These micro frontends were broken down following the similar domain-driven design principles that we applied to our microservices. The result was a range of cross-functional teams building their own frontends and microservices, rolling out independently, and a product ready to be deployed with whichever squad is ready to go!
Easier said than done, right? For starters, the idea of integrating multiple JavaScript files, ensuring that stylesheets aren’t being overwritten, dependencies aren’t being loaded multiple times and most importantly determining when to load what in the browser is daunting. All of these problems were absolutely pivotal for us to solve before we explored further on this idea. Luckily enough, at that time, single-spa framework had started to gain traction in the product community.
Below is a high-level representation of the final design which is now enabling a vertical architecture end-to-end with full stack engineers working in their squads solving for their business front end to back end, without having any external team dependency.
Single-spa: the framework
At a high level, single-spa is top level router that downloads and executes the code for a route when it is active. The framework for single-spa streamlines the process for integrating micro front ends. The idea is for different front ends to build their own JavaScript bundles, and these bundles can then be fetched to be imported into the index.html file that’s being served in the browser.
These bundles are served responding to a route — for example, if your web app is hosted at www.abc.com — then single-spa takes configuration to show JavaScript bundles that should be served at the base route, such as the home page and navigation bar. For www.abc.com/123, you can serve another set of micro-frontend(s) and so on for www.abc.com/456. You can obviously have some micro-frontends that should be always shown at all routes through single-spa configuration, such as navigation bars, header and footers.
Generally, page layouts represent multiple micro-frontends (from here on referred to as an Application) rendered on the browser. One Application is the main application responsible for most of the user interactions. Other applications serve as navigators, headers and footers which depend on how we want to structure it. An example from one of our internal websites is that we have navigator + header and footer as one app and then a dashboard as the main app on the home page. The dashboard has a bunch of tiles and the navigation bar has a range of links. When the links are clicked through, a new route will be opened on the website, with the main app now being replaced by that particular route app and navbar, header and footer intact.
There are some really awesome features like import overrides which lets engineers develop against different environments from their local workspaces, giving them a really fast feedback loop. Essentially an engineer can go to a non-prod environment and update the import map to pick up the frontend bundle from local rather than from a S3 bucket. This allows them to see how local code is going to behave against that environment without having to push through changes, build the bundle, deploy to a bucket and run it on that environment. This enables a very short feedback loop especially with things that generally take long to verify like — styling, assets, overall look and feel / user interaction and interacting with endpoints from the web app itself. Dev experience is as vital to us as the user experience and single-spa has been able to improve both out of the box.
The pursuit of simplicity
As a team, we’re always seeking ways to remove bottlenecks and simplify everything and we all challenge what we see as waste — whether it be a process, tool, tech or design. In the pursuit of simplicity, we have had a really amazing experience with micro-frontends as it has helped not just deliver business value faster, but also in isolating the problem statements and solving it in a way and shape our engineers know best.
Below is an integrated view of one of the products that we have built where each squad owns an end-to-end vertical slice (domain) of the product. Each of these vertical stacks are applications in their own right and can be deployed independently.
Conclusion
Simplified and modular architecture has enabled our agile teams to deliver at their own pace, choosing the technology and tools that work for them and continuously integrate into the product to deliver enterprise-wide digital transformation.
This has also encouraged us to use similar architecture and design to simplify and consolidate our other services across Macquarie’s different groups, helping to integrate them into a singular product providing a unified experience.
Since we released the first version, using the above approach, we are continuing to roll out more services for staff to access all in the one place. We are always innovating and exploring how we can uplift the digital experience of our people, ensuring everything is Simple, Secure and Scalable.
Interested to learn more? Find out where a career at Macquarie could take you.
