PubSub messaging using Redis and Nodejs

Implementing a pubsub messaging protocol is a common use case in a micro service oriented environment. Among the benefits of implementing such protocol, is creating a loosely-coupled system, that is easy to scale.

In this post we will use an npm package called redis-messaging-manager, that can be used for an efficient messaging solution. We will walk through several examples and use cases.

The Source code for the examples is available at- https://github.com/tomyitav/redis-messaging-manager-examples

Background

Interprocess communication is a common challenge in a distributed environment. Some of the trickey use cases are service discovery, recovering from failures, scaling and more. Using a pubsub messaging protocol can help overcome these issues.

Redis is a key-value store, that also enables a quick pubsub messaging communication between processes. The Redis server can be used as a message broker for linking publishers and subscribers. Here is how it works:

Redis pubsub overview, taken from Mike Wasson, Patrick Fletcher post

We can note that:

• A Publisher need not be aware of its subscribers
• Additional instances or replicas of a service can be created on demand, thus enabling to scale the system.
• New services can be added to the system and use the messaging system, with no changes to the existing deployed services. In other words, publishers and subscribers are loosely coupled.

Introducing redis-messaging-manager

In order to implement a pubsub mechanism, we will install redis-messaging-manager. The library is implemented in NodeJS, using TypeScript. It is designed to have a sane and concise api, but also to have excellent performance. it uses:

• ioredis: a great Node.js Redis client library
• rxjs- A reactive programming library for JavaScript, for allowing easy event processing
• TypeScript- The TypeScript compiler will show errors and warnings on potential problems when using the api.

We will now walk through some of the main features of the library by examples. (The source code is available here.)

• Creating basic pubsub object

import {PubsubManager} from 'redis-messaging-manager';

let messenger = new PubsubManager({
   host: 'localhost'
});
export default messenger;

The messenger object will be used for communicating between different processes.

• Subscribe to messages on a channel- The following code snippet shows how to subscribe to a topic, (or channel) named “redis”

import messenger from "../messenger";

console.log('consuming messages..!');
messenger.consume('redis')
    .subscribe(msg => {
        console.log('Got message- ', msg);
    });

As Simple as that. We get an Rx Observable that we can subscribe to, and we get on next event every time a message is sent on the channel.

• publish on channel- This is how we publish a single message on the channel “redis”:

import messenger from '../messenger';

messenger.publish('redis', 'Hello redis messenger!');

• Publish multiple messages on a channel- it is possible to send a bulk of messages in a single round trip to the Redis server. This is done by using Redis Pipelining behind the scenes, and can significantly increase publishing speed.

import messenger from '../messenger';

let bulkOfMessages: Array<string> = [
    "hello redis 1",
    "hello redis 2",
    "hello redis 3",
    "hello redis 4",
    "hello redis 5"
]

messenger.publishBulk('redis', bulkOfMessages);

Conclusion

In this post, We have seen how easy it is to implement a Pubsub messaging protocol with Redis and Node.js.

I hope you have enjoyed reading the post, and found the library helpful in implementing a messaging protocol in your own systems.

If you liked the post, please feel free to waste a few seconds to give it 50 claps :-)