Tale: Scripts & Services Sharing Code!

10x Engineering Series – Microservices

At Urban Company Engineering, we take a lot of pride in the platforms we build. One of the most leveraged platforms is that of our microservices. This blog is a part of the “10x Engineering Series – Microservices”.
If you haven’t read the introduction, go here.
This blog showcases an example of what-is-possible in our engineering world — based on just 2 simple steps: RPC, Directory of Services/DBs.

“We have grown to be a team known for bold interventions, timed well.”

Think about how your scripts are written and deployed. Scripts are not a favorite for most engineers because of a few common pain points felt.

• I have to write scripts in a completely different stack than my microservices (python?)
• I have to be extra careful while querying databases to ensure my script has the same behaviour as my microservice
• I have had moments when I ran a bad migration

Scripts are not maintainable because scripts are written in a very isolated and independent manner to solve specific problems, the focus is always on getting that work done versus thinking about code structure & design (as the investment doesn’t look worthy to do it an individual script level). Because of the above, testing and debugging is also not ideal for each scripts and hence it further increases the chances of a bug creeping in.

Don’t you wish your scripts were nothing but another code file in your service, you could share functions and code, you could reuse all the validation, and deployment was as simple as “no work”?

Well, that is how we do it.

Our primary stack is TypeScript. Our scripts are in TS. They are in the same repository of the microservice. They can query any file / function of the microservice. That simple!

// SERVICE
/////////// server.js
const RPCFramework = require('openapi-rpc-node');
RPCFramework.initService();
/////////// service.js
const Service = {
  addNumbers: function(a, b) { return a + b; }
}
module.exports = Service;

// SCRIPT
/////////// script_a.js
const RPCFramework = require('openapi-rpc-node');
const NumberService = require('../service');
RPCFramework.initScript();
const Script = {
  run: function() {
    ...
    NumberService.addNumbers(3,4);
    ...
  }
}
module.exports = Script;

Notice that :

• the script initialisation needed no code, just an initScript() .
• reusing the service code was just an import of the service file. (or any other file)

How does this work?

First off, you need to have a stack that extends well to scripts and microservices. Even if you had the same stack (python?), it is not easy to share code.
The biggest hurdle to cross is that scripts need to make connections to often secondary databases. Scripts are often run separately to microservices to avoid the microservice’s performance from getting hit. For code-sharing to work, you need to have a way to load all connections for the script and the service code separately. This is only possible if your code decouples initialisation from the business logic.

Well, we know how to do that from our previous reads.
– Create Central Directory of Services & DBs (read more? …)
– Create Dependency Config for Service & Scripts (read more? …)

2 files needed to run a service — central directory & service dependency config.

Just like we leveraged a dependency config file to automate all boilerplate for setup, we can create a dependency config for the script. If the script uses code from the service, add those dependencies (secondary databases?) as well. The dependency config will evolve to having a section for service, and a section for scripts.

{
  "Service": {
    "MySql": [ ... ],
    ...
},
  "Scripts": {
    { 
      "script_a": {
        "MySql": [ ... ],
      },
      "script_b": { ... }
    }
  }
}

These are the steps:

• Your central Directory of services and databases needs to store connection information for primary AND secondary databases
• Your Dependency config needs to store the script and the associated dependencies. It needs to have a way to tell that a database dependency is loaded in primary mode or secondary mode
• Your platform code while starting up a script, or a server, needs to incorporate the whole primary vs secondary database

Hence, all you need to do it give scripts it’s own initialisation .. and boom!

What More?

Right now, you have to explicitly call the dependencies needed for your script, even from the microservice.
Suppose you need to query microservices quite often. In fact, you want to query other microservices, but not disturb their production primary databases. What you really need is to launch the microservices just for the script, and connect them to secondary databases instead.

This is an easy extension to the platform. We at UC don’t do this yet, but worth sharing the potential.

You can change your dependencies in scripts to include other microservices. And while running the script, you can load ALL the required microservices (connected to secondary). This way, your dependent microservices (application server and primary database) are isolated from your scripts.

{ 
  "Scripts": {
    { 
      "script_a": {
        "MySql": [ ... ],
        "RPC": [ ... ]
      }
    }
  }
}

All this requires no extra service or script code to be written. All this is simply powered by the magic of platforms and configs.

Summary

Writing scripts hasn’t been easier.

“Zero boilerplate no matter what the complexity!”

Next >> Tale 3: One Line To Monitor Them All!

Sounds like fun?
If you enjoyed this blog post, please clap 👏(as many times as you like) and follow us (@UC Blogger) . Help us build a community by sharing on your favourite social networks (Twitter, LinkedIn, Facebook, etc).

You can read up more about us on our publications —
https://medium.com/uc-design
https://medium.com/uc-engineering
https://medium.com/uc-culture

https://www.urbancompany.com/blog/humans-of-urban-company/

If you are interested in finding out about opportunities, visit us at http://careers.urbancompany.com