Hydra Microservices for the ExpressJS Developer
For many NodeJS developers building Microservices is really just a matter of using ExpressJS and limiting the number and scope of their APIs. So rather than put all of their APIs into a monolithic Node app, they simply create a bunch of smaller services - each with its own dedicated API. Get fancy and add containerization and then call it a day.
However, as their number of microservices grows, so does the number of intermediate solutions used to bind them together. Add more than one microservice and they’ll need a way to route messages to each. So, enter Nginx to address that concern. Perhaps, some of the microservices need to communicate with one another? Hmmm ok, use HTTP Rest calls between services? Or perhaps use a message queuing service such as MQTT, RabbitMQ or Kafka. How about monitoring microservices? Well there are a lot of solutions there! Overall, each concern requires you to learn a new technology and write some code to bind it all.
The great news is that there are no shortage of solutions. The only real shortage is your time. But as I used to say back in the day… the night is still young.
In truth, microservices are no joke. And this is true of distributed applications in general. A challenge is that code isn’t all in one place — and instead must communicate across a network.
The added challenges may not seem like a huge issue for teams with extensive experience. But what if it’s just you, or the team you’re in hasn’t really done this sort of thing? How does an experienced ExpressJS developer build robust microservices?
Hydra is a single NPM package designed to address the above concerns and much more. With Hydra, you can build, configure and deploy microservices in record time. There is however, one small catch… You’ll need an instance of Redis. Given that over the years Redis has been voted by developers as one of our most loved and wanted databases it’s likely you’re already using Redis or know someone who is. If not, no worries — you’ll be using Redis by the end of this post.
In this post I’m going to assume you’re comfortable with NodeJS and ExpressJS but perhaps have little experience building complex microservices. If that’s not you, then play along — there’s sure to be some interesting bits you can take away.
Oh, two points before we begin. The best way to read this post is to quickly skim through it and only try the provided instructions after, and only if this post intrigues you. The second point is that this post might seem really long… but that’s only because I’m big on visuals.
Setting up Redis
Earlier I mentioned that Hydra requires Redis. If you already have access to a Redis instance you can safely ship this section.
By far the easiest and most useful way to launch Redis is simply by installing Docker and running Redis as a container.
See this quick start guide and you should have Redis up and running in under 10 minutes. And… there are tons of other guides to getting started with Redis.
Show me the code!
At this point you should have access to a running Redis instance. And now we’re ready to play with some code.
Since I value your time, I’ve created a Github repo to spare you from typing in code.
Enter: git clone https://github.com/cjus/hydra-intro.git to pull that repo.
Execute the stackup.sh file to launch a test docker cluster running Redis. You can have a look at the compose.yml file to see the Redis specifics if you’re interested.
All of the code examples are in the examples folder. In prep, go into the examples folder and run npm install
Let’s begin by reviewing a simplistic ExpressJS example. Granted, it may have been a long time since your express app was this simple.
The Hydra version is a tad bit longer so let’s step through it.
We begin by loading hydra-express on line one. HydraExpress is a Hydra and ExpressJS binding. So when using HydraExpress you don’t need to also include ExpressJS because HydraExpress is already doing so. Using a reference to Hydra we ask it for its reference to ExpressJS on line two.
On line three we obtain a reference to the Express router.
On line 6 we initialize HydraExpress using an external config file which we’ll have a look at shortly. The init function requires a callback as its second parameter, which will be automatically called when HydraExpress is ready to register your routes.
We use the ExpressJS router reference (api) on line 7 to define a root route. Then on line 10 we simply register the route.
Scroll back up and compare the two examples — they’re really quite similar!
Let’s have a look at the hello-config.json file referenced on line 6.
This is a standard Hydra config file. It begins with a hydra key pointing to an object. We define our serviceName and servicePort. The serviceIP and serviceType are optional. The really important key is the redis key on line 8. Here we tell Hydra that Redis can be reached on the local machine on port 6379. The forward slash 15 specifies which Redis DB it should use.
If you’re not that familiar with Redis - don’t worry too much about the above configuration. You can learn more about it later as you try to point to remote Redis servers.
So we’ve compared a simple ExpressJS app with a HydraExpress one. The Hydra version is definitely a bit more work — requiring us to type six more lines of code and use an external configuration file. Granted in a more complex project, those minor differences will be the least of our problems. That said, what do we get from the HydraExpress version that we don’t get with a basic ExpressJS app? After all, both examples do the same thing when accessed via http://127.0.0.1:8080
To begin answering that question, let’s take a look at a second, slightly more complicated Hydra example named hello-inst.js
The code adds a single line (#2) and changes an existing line (#9) in the earlier hello.js example.
Line #2 asks hydraExpress for a reference to its internal hydra-core module. Using that module we gain access to our service’s name and its instanceID on line #9. We’ll see those changes in action shortly.
Let’s go ahead and run this example.
And let’s access http://127.0.0.1:8080 from a web browser.
Here we see that, as expected, the example returns its serviceName and instanceID.
Every running instance of a Hydra-enabled service has a unique instanceID. The ID is used by Hydra for all sorts of tasks ranging from presence management, messaging to routing.
Ok, let step this up a notch! Our Hawaiian brothers and sisters have a Pidgin saying called Howsit. Translated as “how is it” or “how are things”, or “How are you doing”. Often heard as “Eh, braddah, howsit!”. Our next app is aptly named: howsit.js
This version is the same as our last example, minus a config change on line 7 and a cosmetic text string change on line 9.
The more interesting change occurs on line 5 of the howsit-config.json file, where we changed the port from 8080 to 0 (zero).
When Hydra encounters a zero port number it automatically assigns a random port number greater than 1024. This is useful when you need to run multiple instances of your service on the same machine — as port conflicts can be avoided.
If we run the howsit service we see:
Note the new port 34310 above. Accessing http://127.0.0.1:34310 returns: Eh, braddah, howsit! howsit — eaf6e4e1ad0242cf96b3f3407d50120c
Cool. Now open additional terminal windows / panes and run multiple instance of the howsit service:
Each can be accessed via its assigned port number. However, if we take a moment to install the hydra-cli command line tool we can see some interesting behavior!
We do need to configure hydra-cli before using it. Namely to tell it where Redis is located. We’ll use the same values in our config files.
We can ask hydra-cli for a list of all running services (nodes):
So Hydra is well aware of our running instances!
We can use the hydra-cli REST interface to access our services. The syntax reads: rest servicename:[http-method]api-path
Here we see on line 6 that we received a response from instance b0c17930878548669ffd56713f3d8992 Note that we didn’t specify a port number!
Repeating the hydra-cli rest command a few more times returns:
“result”: “Eh, braddah, howsit! howsit — 8172eea68f4a4d0599225725cb410572”,
“result”: “Eh, braddah, howsit! howsit — eaf6e4e1ad0242cf96b3f3407d50120c”,
So requests are being load-balanced between our available instances.
If we stopped one of our instances and reissued the hydra-cli rest command we’d see responses from only the available services. That’s because Hydra manages presence and knows when an instance is no longer available.
I’m sure you’ll agree that we’re now on an upgraded ExpressJS train!
At the risk of creating a even longer blog post, I’d like share a bit more…
Distributed services often need to communicate with one another. Using Hydra this becomes surprisingly simple. In our next example we’ll look at two services: send.js and recv.js to see how services can communicate.
First. let’s take a look at the recv.js example. Note that I’ve removed the express routing bits since they’re really not relevant to our example.
On line 6 we setup a hydra message handler and simply output whatever message arrives.
The send.js example is a lot more interesting. Here we grab the serviceName and instanceID and use them to identify the sender on line 6. On line 7 we use the hydra createUMFMessage call to build a message object using the required, to, from and body fields.
Note that on line 8 we specify that the message should be delivered to a recv service.
We then take the greetingMessage and send it via the hydra.sendMessage member function. Yes, sending messages is really that simple.
Now let’s launch multiple instances of our recv service and one send service.
Every time we start and restart send, a message is sent to one of the two recv services. As we saw earlier, messages are load-balanced based on presence.
The really cool thing here is when services are running across networked machines — no code changes. Hydra knows where each service is running.
There are many more features we could look at, such as message queuing, health and logging, but that would require a much longer post. Hopefully this gives you a sense of just how easy it is to use Hydra to build microservices.
Next steps
Earlier we saw that there’s a handy hydra-cli program. There are actually quite a fews tools available. One tools is called the HydraRouter, a hydra-powered microservice which can route incoming HTTP and Websocket messages to any of your services.
Hydra was open sourced at the 2016 EmpireNode conference and powers the FlyAnywhere live video streaming service. It’s in use by an increasing number of companies, and might be a great fit for your next project!
Checkout the project documentation at HydraMicroservice.com
For a deep dive into how Hydra actually works see my RedisConf18 presentation.
—
Thanks for reading! If you like what you read, hold the clap button below so that others may find this. You can also follow me on Twitter.
