High Performance Messaging with NATS

NATS is a simple, secure and high performance open source messaging system for cloud native applications, IoT messaging, and micro-services architectures. Whereas NATS Streaming is a data streaming system powered by NATS, and written in the Go programming language. This post will present an example using some patterns from both platforms.

Introduction

Communication in distributed systems can be complex and difficult. NATS was built to meet the distributed computing needs of today and tomorrow. It is a simple and secure messaging system made for developers and operators who want to spend more time developing modern applications and services than worrying about a distributed communication system. NATS comes in two variation:

1. The NATS Server offers an at most once quality of service. If a subscriber is not listening on the subject (no subject match), or is not active when the message is sent, the message is not received.
2. NATS Streaming Server is great when we need message persistence as it provides at least once delivery. From an architecture point of view, NATS Streaming server is a NATS’ client which adds persistence, message replay and durable subscriptions to NATS. For more detail, refer official documentation.

Use Case

Let’s imagine we have a data collector application. Several agents are located at different locations which gather and process data from different devices. In this article, we will talk about start agent and get agent health workflow using Go and Python clients, but clients are also available for many other languages.

NATS & NATS Streaming clients

Let’s now set up different components involved.

Running a NATS Streaming Server

We can use official NATS Streaming Server image from docker hub and run the following command

docker run -p 4222:4222 -p 8222:8222 nats-streaming -store file -m 8222 -dir datastore

Pub-Sub Pattern Implemented with NATS Streaming

Publish/Subscribe (1:N)

Let’s say we have to start the agent and if an agent is offline, our message should not drop. NATS Streaming Server is built to handle this. The following snippet shows the publisher code implemented in GO where the manager application publishes a message (containing the event type and configuration) to the “agent.1AX24” channel.

On the subscriber end, our agent is listening on the “agent.1AX24” channel. Additionally, start position is set to last received so that if an agent gets offline, the last message from the channel is used for processing. The following snippet shows the subscriber code implemented in Python.

Note: Nats Streaming Subscriber has many options such as StartAtTime, StartAtTimeDelta, DeliverAllAvailable for which you can refer to official documentation.

Request-Reply Implemented with NATS

Request/Response (1:1)

Agent Health monitoring can be done using request-reply pattern. Manager application will send a request on “agent.1AX24.health” channel and will wait for the reply for 5 seconds. If the agent replies back in 5 seconds, manager will consider it healthy; otherwise the agent will be considered dead. The snippet below shows the manager code.

Agent will subscribe to “agent.1AX24.health” channel and will wait for the message. When a request is received by the agent, it will reply back with a success message indicating that agent is healthy. Python implementation can be shown below.

Summary

In this post, a quick introduction to NATS and NATS Streaming Server was given. If you too want to experiment with NATS, you can go grab the stan docker image and start playing around to learn more about it.