An easy way to make an inclusive proxy in NodeJS

Managing microservices interactions properly

Introduction

In Wolox we have a wide variety of projects we are developing. The biggest one, in which I am currently working on, it’s a marketplace app. It includes users, products, dispatch flows, finances, purchase orders and logistics modules all involving 6 API’s interacting with each other in a microservice architecture. Each of them with their own database. These APIs have different technologies- some of them are in Ruby on Rail + Postgres, others in NodeJS + MongoDB, and we are starting a new one in Java.

As you can imagine, in the web application some views require information from multiple services, and some events involve multiple validations and data modifications in different API’s. Managing these kinds of requests in frontend has its own, extremely frustrating, complexity. In order to avoid this, the architecture includes a proxy which we call BFF: Backend For Frontend.

Backend For Fronted started as a simple proxy. There was a controller for each resource of each API. Each controller made an export for each method of the resource, which was in a config file. This is how they looked:

And for each route, we also had to declare it in a routes file.

The pipe helper was a just a generic request and some logs we added in order to see headers and bodies.

The first functionality we added to BFF was the authentication. The API’s require a user token, and for this, we needed to check the users API if this token was correct. This needed to be done for almost every endpoint, so we added userAutheticatedPipe. . Later we did the same for admins.

If an endpoint needs data from different API’s, we do it in the controller.

Problems

We started facing some problems with this implementation. First of all, we were repeating the first gist of the post on every controller of our app, and every time we added a new resource, we had to add that logic. This was not maintainable at all. Any change in our serviceRoutes file will break every single endpoint and force us to change every controller (at that point we had 20 of them).
The serviceRoutes file started to be too verbose. This is arguably because it clearly reflected which resource depended every route, but adding a simple route was too much trouble in a +300 lines file.

And the biggest problem: every time someone from another backend tech had to add an endpoint, he/she asked us. Endpoints that we did not always have all the information about how they worked and what they were supposed to do, and we had to test before making the pull request. The NodeJS team of this app would not tolerate this for too much time.

The solution

Having some time left in a sprint, we started to clean our tech debt with this problem. The challenge was to stop repeating code and to make the endpoint addition in BFF easy enough for any developer to do.

The serviceRoutes file should be really simple: a config JSON and a key to add the authentication function depending on the endpoint. A piece of code to show how to add all the endpoints in the serviceRoutes with their proper method, route, and authentication. After a few attempts and code reviews, we achieved to make it work.

It is an abstraction of what we repeated on every controller. Also, we added the default method GET so we do not repeat that constant on every endpoint. For another method, we added in the methods key.

We wrote in the README of BFF the steps for adding an endpoint.

So now we have an inclusive proxy for our application! EVERY developer of our team can add an endpoint (and also may feel that he/she knows a little bit of NodeJS, although this may not be true). No more messages, cards or tickets assigned to the Node team asking to add an endpoint. And no more blocking feature because of this.

Conclusions

Besides the technical resolution and implementation, the most important thing I learned from this is to dedicate time to refactoring. We all know that features are always more important than this and, as a backend developer, I am used to having that feeling that nobody really sees the benefit of doing this kind of stuff the right way. With BFF we are not only happy with the result but,it improved our daily workflow making it more agile. If you really have a tech feature which can make a difference, don’t keep it yourself ,share it! Let others see and understand the benefit of implementing it!