Tale: Zero Boilerplate Services!

10x Engineering Series – Microservices

At Urban Company Engineering, we take a lot of pride in the platforms we build. One of the most leveraged platforms is that of our microservices. This blog is a part of the “10x Engineering Series – Microservices”.
If you haven’t read the introduction, go here.
This blog showcases an example of what-is-possible in our engineering world – based on just 2 simple steps: RPC, Directory of Services/DBs.

“We have grown to be a team known for bold interventions, timed well.”

As engineers, we might believe that the lesser the boilerplate code, the lesser the chances of errors. However, have you wondered how “less” could “less” be?

Let me show you the amount of code a repo needs to have to be production ready at Urban Company.

/////////// server.js
const RPCFramework = require('openapi-rpc-node');
RPCFramework.initService();
/////////// service.js
const Service = {
  addNumbers: function(a, b) { return a + b; }
}
module.exports = Service;

That’s it! There is zero boilerplate code. Zero code fat.
(well, just the one line for service initialization – even that could be removed!).

Don’t believe us?
Let’s add a little complexity. Let’s query a database to get a number (assume it’s saved already).

/////////// server.js
const RPCFramework = require('openapi-rpc-node');
RPCFramework.initService();
/////////// service.js
const RPCFramework = require('openapi-rpc-node');
const Singleton = RPCFramework.getSingleton();
const NumbersDb = Singleton["mysql_numbers_db"];
const Service = {
  addNumbers: function(a, b) { return a + b; }
  getNumber: function(a) { return NumbersDb.query( ... ); }
}
module.exports = Service;

Notice, there is zero code to connect to the DB. It is already there, magic perhaps?

Let’s add a little more complexity. Let’s query this “number service” from another service.

/////////// server.js
const RPCFramework = require('openapi-rpc-node');
RPCFramework.initService();
/////////// service.js
const RPCFramework = require('openapi-rpc-node');
const Singleton = RPCFramework.getSingleton();
const NumberService = Singleton['number-service'];
const Service = {
  getSecret: function(a) { return NumberService.getNumber(a); }
}
module.exports = Service;

Are you amazed yet? Yes, this is the only code that needs to be written. It can run in our production as is.

( In practice, we tend to add a few more things to the production code to make it maintainable. Top two being – typed interfaces and a good file/folder structure. )

How does this work?

The prerequisite to understanding the how is to have done the 2-simple steps:
– Create RPC
– Create Directory of Services & DBs

Once you have a central directory that lists all the services and DBs, you are just 1-step away.

You just need to create a simple file per service – Dependency Configuration.

This file will list down all the dependencies the microservice needs. It will look something like this:

{
  "MySql": ["mysql_users_db"],
  "RPC":   ["payment_service", "request_service"]
}

This file could be stored within the service against a specific name, say dependencies.json.

Each of the keys mentioned in the lists is already there in the central directory of service & dbs. ( Read up on how to do this here. )

In your framework code, all you need to do to initialise everything is :

1. look up a file called dependencies.json
2. query the directory and retrieve the credentials for each key
3. use the respective “init” functions to instantiate a connection
4. store the connection in a singleton object so that files can access the connection (or pool)

2 files needed to run a service – central directory & service dependency config.

You will realise, the “singleton” object is the only state in the code. The actual codebase in the repo is completely stateless – no instantiation needed. Hence, you have cleanly separated the state and the stateless part.

What More?

At a certain scale, you will want to have service specific requirements when making connections — pool size for databases, timeout thresholds, etc. The real way to create this dependency configuration is by storing a list of objects, each defining the parameters to connect to that server.

{
  "MySql": [ 
    { 
      id: "mysql_users_db",
      .....
    }
  ],
}

Zero Code – Threads & Timeouts for Databases

Connections between two servers often require tweaking of a lot of parameters such as thread pools, etc. For example, connecting to databases via initMongoConn() / initMySqlConn() will need a lot of parameters , like – thread pools, primary vs secondary database, timeouts, etc.

Zero Code – Event Queue Connections

You can extend your platform by pre-initialising code for pub-sub from an Event Queue.

This helps in standardising the queue infrastructure used and also the way to use it. Will need to do the following :

1. Add the ability to connect to the event queue in your framework initQueueConn() .
2. Using the dependency config, add the queue connection parameters. Further, specify the event names you’d want to consume and publish (either here, or separately). This way, your platform code can automatically register what your service needs to consume and publish.
3. Create a standard API in your repository that allows you to consume an incoming event. This basically simplifies even “event based” data sharing to simple API based implementation. API /consumeEvent implemented by Service.consumeEvent(eventName, eventData) .
4. Create a library function for publishing of events RPCFramework.publishEvent(eventName, eventData) .

Zero Code – Circuit Breakers / Timeouts for External API Calls

You can build a resiliency layer for free. Instead of writing code to break the circuit / timeout in every service or file that connects to an API, you can simply write a config and let the platform take care of it.

This requires you to wrap ALL external API calls within a wrapper perhaps callExternalAPI() . This way, you can put the ability to timeout inside this.

Summary

This list of what more could you standardise is endless. In summary, you can achieve a lot when you start to standardise your stack and build platforms on top. Platform building is never a short-term leverage. Real benefits unfold much later – just like our two simple steps led to this.

“Zero boilerplate no matter what the complexity!”

Next >> Tale 2: Scripts & Services Sharing Code!

Sounds like fun?
If you enjoyed this blog post, please clap 👏(as many times as you like) and follow us (@UC Blogger) . Help us build a community by sharing on your favourite social networks (Twitter, LinkedIn, Facebook, etc).

You can read up more about us on our publications —
https://medium.com/uc-design
https://medium.com/uc-engineering
https://medium.com/uc-culture

https://www.urbancompany.com/blog/humans-of-urban-company/

If you are interested in finding out about opportunities, visit us at http://careers.urbancompany.com