Why I moved from Monolithic Backends to Microservices.

We all have that first project that sticks with us forever, well mine was a small API designed to handle low load traffic for my school. It handled cases and student attendance. It never saw the light of production due to me moving out of the area, but it was enough for me to learn about a platform called “express.js”. I had just dived into Node.js.

Monolithic, Microservice? You’ve lost me.

Before I get too far ahead, let’s stop and take a moment to talk about what exactly monolithic and microservices are. Monolithic essentially, in this use case, means that everything is within the same stack (think of a stack like a sandwich). An example of monolithic is easily represented in a diagram:

Image from https://www.thoughtworks.com/insights/blog/bff-soundcloud

Every platform polls this one massive backend, this is usual the start of projects, and has some advantages. Everything is isolated and can be hacked upon. Programming languages, and kernels also have sort of the same concept. One example of a monolithic stack is the Linux Kernel by Linus Torvalds. The next design type covered here is the Microservices architecture. The basic concept behind microservices is taking a massive web-app, API, whatever, and separating it into small components and having each component interact with one another.

http://hardmockcafe.blogspot.com/2016/01/avoid-vertical-limits-in-microservice.html

Depending on the design, the cart service could be scaled to have multiple instances as well as the catalog service. Each service is then communicated with via the API gateway. Then, another method of “servicing” can be done in the UI, but that isn’t really brought up here. Each service is independant of the other, and uses some sort of message queue, which we’ll go into later, or other method of “Inter-Service Communication”.

I tried to make a giant monolithic web app, it didn’t work.

Moving back to my design architecture and fast forward a couple of years, I began working on larger projects, such as NodeMC/CORE. While these projects weren’t much different, and they still used Monolithic backends, they did demand a higher amount of scaling and ease of development. At this point I designed a way of dynamically constructing express routes (Originally designed at verteilt/backend).

Creating Express Routes Dynamically by Using Classes.

By utilizing both async and fs, we could easily construct classes and provide a express#Router object to the new router, as well as pass necessary variables. A route object looked something like this:

Dynamic Routing with Express.js

While this approach worked well for the project, it doesn’t work well for large scaled multi-lang environments. The code is still monolithic which causes issues for large scaled development teams (merge conflict time!), and everyone needed to program in JavaScript. This is when I realized the project needed to change: Introducing Verteilt.


While I won’t go into much detail about what exactly Verteilt is, however I will say that Verteilt is a service that needs to be able to scale, scale, scale! However, it needs to be able to do so easily and at some points not at all. It should be able to handle no traffic, to trillions of requests per second, while all being able to scale with no hassle. This was a relatively new concept to me. I had gotten a lot of interest in the idea from other developers and had generated a small developer team. This time I had to think ask questions like; “How do you scale parts of an API?”, “How do you handle parts of an API that need to be scaled more than others?”, “How do you modify all of the API with potentially hundreds of developers in production”, and most importantly: “How do you make it cost-effective?”.

How the hell does the real world do this?

The first place to look was @twitter and @netflix. Twitter and Netflix both use a microservice based architecture, as well as things called Message Queues, and Job Queues. This was the first time I had ever heard of things like this. A way of thinking of Message queues for monolithic apps is easily comparable to the Event Queue in Node.js (though they aren’t strictly the same thing at all), you can create ‘topic’ and post information on it and have the listener consumer that event and there you go. Job queues are really just a way of solving the problem of having large amount of inflow but not a lot of processing power. Or even just the problem of having a language being single-core (like Node.js!). By having a job queue, you can evenly distribute work to workers. You can even run multiple on a single machine per core, allowing you to get passed limitations of languages.

A job queue isn’t a very hard concept.

Netflix had some very interesting side effects of using microservices;

  • Testing — Could randomly kill machines in production to find errors in the architecture design.
  • Stability — When one service crashed, everything else stayed up.
  • Versioning — Each component could be maned by a different development team.
  • Languages — You could have one micro-service in node.js and one in ASM

At this point I had been sold. My team is a very diverse one, some of us speak JavaScript, some of us speak PHP, and some of us even speak Ruby. The ability to develop microservices in other languages and just use bindings for Apache Kafka and other Job Queues on top of Redis was what we needed.

Adapting a Monolithic App to Microservices

Moving from one to the other wasn’t very easy. There is a tremendous amount of new concepts to work with and learn moving over to the microservices architecture, and the sheer volume of templating. A lot of microservices success, as a design, comes from templating each service. Each service should have template to easily deploy new services with i.e; metrics, languages, frameworks etc. We did this both code wise and platform wise. Using Docker to isolate systems, and just writing libraries to interact with our stack.

One of our greatest problems was that a lot of our team is used to using Express.js, we solved this by implementing a subset of the express.js API and warping that around Kue, which would have the response and request transparently forwarded through the service chain.

Verteilt Architecture

Another big issue is keeping track of requests and responses. Because our product is currently a REST API, we had to track and allocate sockets. This is a big change from the monolithic design beforehand. Instead of just being able to use res#send, it had to find the socket and it’s Express#Response object from memory. This was solved by tracking request through a unique id, as well as a unique gateway id to solve determining which gateway to send it out of. This allowed scaling the gateway and for a request to travel the entire service “network”. Along with tracking the response object we tracked the state allowing for time-outs to be recorded and added to metrics to track services taking to long.

Was it all worth it?

That’s something I cannot answer yet. I still have a lot to work with, such as streaming JSON and other things over this new architecture, and even getting other languages to work in this new stack. It’s an exciting time to be a backend developer and I’m looking forward to the challenges and benefits of using this new architecture. It’s why I code.