How we split the Monolith. A React and GraphQL success story.

How many times did you start a job and had to deal with legacy code ? Code without tests ? Code that “has been working for few years now just fine”, but now you have to modify to add a functionality ?

I once had to deal with a ~400k lines of a single application written in Javascript and Ruby on Rails in a mixed mostly single page app: some pages were managed by Rails, other were initialized by a Rails page, but then had all the business logic on the browser.
The best part was the templating system: a custom made Handlebars + Jade + ERB. This configuration resulted in files which looked like html but had variables which were resolved by the server (ERB) some others at Javascript bundling time (Jade) and other variables which were resolved at runtime (Handlebars).

Definitively working in that environment was not a pleasure and since it was part of a big contract just started, i wasn’t up to give up so easily.

I was on a mission: I started this contract because i wanted to have an impact, here was my chance to prove it.

Understand the Product

First thing i did was understand how users were using the product. To do so I observed how highly trained people in the company were using the product. It was a naive approach but it made me understand the core functionality expected by the user in a small amount of time. Analyzing usage logs was also another insights of what were the steps that the users were taking while using the product.
These crafted the critical user journeys. By creating tests around those user journeys i had a pretty solid canary that would alert me if a change was disrupting the steps our users would make. I was finally able to do significant changes to the codebase and be sure that i was not altering the user experience.

Understand the Technology

I then moved in understanding the technology. It is a fundamental step to spotlight areas that could be the beginning of the evolution on the app itself. In my case, one of the parts that needed more love was the templating system of the web app, alongside with the architecture that simply didn’t evolve from quite a while.
Codebase was mostly written in CoffeeScript, a language I didn’t know but quickly picked up. At the time in which ES6 and Babel were doing great progresses, using CoffeScript felt like a step backward in the set of knowledge that will help me in my career.

You could see a lot of pieces of code that would deliberately have an in house solution while the community had well established solution: i saw in house implementation of Lodash and Backbones and started thinking what technical reason would have justified that. Didn’t find any.
I started thinking that the problem was mostly the team culture: team preferred in house solutions and doing so didn’t keep up to date with the progress by the web community. React, Redux, D3 were all alien technologies in this environments and try to introduce could easily upset some team members.

Evolve, never rewrite

An organization is never going to dedicate time to rewrite code, especially code that works and satisfies the business needs. So the alternative to stagnation is Evolution.

Like in the animal kingdom, new species come from alterations of existing ones. My alteration of the codebase was introducing React.
I build a Backbone View that was able to instantiate a React Component inside it, it would get its props from the Backbone Model. I knew it was not a long term solution but gave me enough space to demonstrate to the team how React components were easy to write and test. As usual with any change some team members were not pleased by it, or would have done differently. Promises of improvements were easy to make and the advantages demonstrated were already enough for the team to move forward with the adoption.

In a relative short period of time the adoption of React wrapped in Backbone Views changed the team dynamics. We could advertise that we were using React, that brought us some good team members in the team, which definitively changed the team attitude further towards established community software.

At this point new project came in. The UX changes were so vast that felt like it was time for a broader challenge.

If coding is art and painting is art, code is like a painting. Imagine you have a completed canvas of a cold night sky and you want to add a blazing sun. You better off starting with a new canvas than trying to adapt the existing one.

That is exactly what happened. I started convincing my higher in rank that we were finally technologically ready to start fresh by making new web applications, starting from scratch with a React/Redux architecture. It went well. The React alteration was evolving into a new creature.

Team members started being excited: we created a new vanilla javascript codebase with just React and Redux in it. Code started pouring in, discussions and article sharing among team members went up, team morale went up. We reached one of most productive quarter in the company in terms of feature shipped to the end user and most importantly, we hit all our aggressive deadlines.

It was hard at times. I learnt the difference between simple and easy. Some team members would just take the easy route: copy-pasting CoffeeScript code into the new codebase to be removed later. Probably we would have never seen the to be removed later part happening.
Reasoning and re-scoping of the features helped team members to make simple instead of easy decisions that would have materialized in tech debt later on.

When the deadline hit, we were ready. Release went smooth. Usually deadline for big projects like this one would shift several weeks due to last minutes bugs or technical difficulties, but it was not our case. Team did an amazing job. We were engaged into what we were building.

Momentum

Matching the deadline really give us visibility in the organization. From the bottleneck team, we became one of the most productive teams. It gave us leverage and Momentum.

Momentum is one of the best leverages a team can have. By demonstrating the ability to Commit and Deliver on Deadlines we proved to the organization that we could be trusted.

Convincing the organization to make broader, more aggressive changes suddenly became easier. Other members, engineers or not, were trusting us.

We started thinking how we could keep the Momentum going and provide value to our Company and ultimately our Users.

Microservices

We were now dealing with new web apps, in a new codebases, but still talking to the same monolithic Rails backend. Could we use the learnings we just had in web apps for backend oriented applications ? Instead of avoiding to keep the Monolite be a core part of our development, could we adopt a similar strategy to evolve ?

We started by auditing our own code. We tried to analyze the code and map to product features. Once we had that list, we were able to group those features in clusters or areas of competence. Those areas were for example: User Management, Projects Management, User Licenses and many more business specific which i’ll not go into details. We could spot that each of those areas could be deployed independently, some didn’t change in years, some needed to change at some point to enable new features.

Eventually these new feature requests came. Changes were supposed to be big in the data model we provide to our users. After some thinking on the features to provide the path was clear: technology had to evolve to support the planned product evolution.

We started with a new service to support those changes, soon they became two. This time also backend engineers were involved in the process. Change brought down team barriers as well: backend and frontend engineers started to collaborate more. More services meant also that agreements or contracts between those services and ultimately with the frontend apps.

GraphQL

Here is when GraphQL came into place. Some team members knew the principles behind it, but never had chance to try if not in personal hobby projects. The problem to establish binding contracts between backend microservices and frontend apps seemed to be solved well by Apollo Server, a Node based GraphQL server implementation.

The introduction of this component made us able to define contracts between services and frontend apps in code, with multiple reviews from different teams. No more shared document with json snippet being passed in meetings or email chain. The contract was the GraphQL schema of the logical elements that each service exposes via its api. Since the schema is typed, we could easily have GraphQL return mocked data. We were able to develop backend and frontend parts independently without waiting one another. This would probably have taken a considerable amount of more communication effort to achieve without GraphQL.

GraphQL gave us the ability to see the data provided by our microservices as a single service with the schema as our contract.

This allowed us to build microservices that were responsible from logical entities we exposed in our product, GraphQL would solve the relations between those logical entities, requesting data from many microservices, and returning them back to the client as a whole.

Success and Beyond…

All this happened in around a year timeframe, with an handful of team members. After this period, a project built based upon 6 to 4 years old technologies switched to a full React + GraphQL + Microservices architecture and new codebases. During this year we kept shipping incremental changes, around bigger milestones of the product, the company grew significantly.
A handful brave, smart and efficient team members did successfully renovated a company from within, improving velocity and future products development.

And it all started with a simple React component, by a single software engineer…