ZeroMQ with Kotlin: part 2, a bit of push-pull and pub-sub

In part 1 we saw how to create a simple server using Reply and Request pattern using ZeroMQ. We used the Java port of the library which worked wonderfully in Kotlin by adding this to our dependencies in our build.gradle file

implementation "org.zeromq:jeromq:0.5.0"

We mentioned in passing other kinds of sockets we can make using ZeroMQ. We will cover a simple example of a Push and Pull pattern using zmq.PULL and ZMQ.PUSH and also cover one for Publish and Subscribe with ZMQ.PUB and ZMQ.SUB. The content will be very similar to the one from the previous tutorial and we will just tweak a few values to have the example up and running.

The basic idea with Publish coupled with Subscribe and Push with Pull is all about the flow of information, where we have one or more nodes Producing/Pushing out information for example a temperature sensor sending out data, a smart-wear gadget sending health data say like hearbeats from a smart watch, a drone or a even the self-driving car you might be designing(who knows you might be the next Elon Musk). On the other end we have a node Subscribing/Pulling information from a source for example a data center, heating system reacting to the temperature change and the like. They’re however subtle differences between Push-Pull and Publish-Subscribe. I won’t outline all of them but i think it’s important to know where they differ to prevent surprise when working with them

One important difference between Push/Pull and Pub/Sub is that Push/Pull doesn’t drop messages when there’s no reciepient. What I mean by this is that if one node sends data through ZMQ.PUSH to a specific client, but for one reason or another the client is off. Push will store the message in a queue, the more data it tries to send out without succeeding will keep adding up on the queue. When the client comes online they’ll receive all the queued up information. On the other hand in the Pub/Sub pattern, if the publishing node sends data through ZMQ.PUB but the subscriber is disconnected, the data will be dropped. Both of these have their use-cases and hence their existance in the library with their different functionaliy. For instance, when working with a drone, we would have a controller that communicates back and forth with the drone. The drone would be sending location and altitude data which the controller could use. What we would want is realtime data to which we can respond to when controlling the drone. In the instance we have small disconnections, we would not want the data to accumulate on the resource constrained drone and then sent suddenly in one burst upon reconnecting to the controller. What we would like is immediate realtime data upon reconnection ignoring the snippets of data during the disconnection period. In this case Pub/Sub serves us better compared to Push/Pull. Push/Pull pattern would come in handy where we don’t want to lose any data or being hit with a large spurt of data doesn’t have any repurcusions to our project, say for example we had humidity detector sensor sending us data, if for some reason the puller of the data disconnects, there wouldn’t be any operational risk in stacking up the data and sending it in one batch to the puller when they come back on — note:depends on what you’re doing, depending on the symptoms of your app you’re encouraged to talk to your doctor :)

Another thing is that PUSH blocks when there’s no Peer ready to receive a message whereas PUB doesn’t. This makes sense as PUB drops the message if there’s no peer while PUSH just keeps pushing to the queue. This is important while writing out your code as you may have some block of code not getting reached below your PUSH code with no puller node, the code blocks until the PULL side comes online. It may also be useful information to help avoid getting surprised when some block of code is reached unexpectedly on your PUB side despite the SUB side not being connected.

Another important thing is that PUB sends each message to the many subscribers(SUB) i.e. if we have a publisher A with many subscribers say B, C and D, they will all receive a message like “hello” when A publishes it. This is compared to Push where messages are sent to one of the many nodes pulling the data i.e. if we have one node A pushing “hello” to nodes B,C and D, only one of them will receive the “hello” message. From the ZeroMQ guide Pub-sub is like a radio broadcast; you miss everything before you join, and then how much information you get depends on the quality of your reception

We’ll have 2 files, MyPull.kt and MyPush.kt. The structure will look something like this

And since i’ve gotten you reading this far i’ll mention something i forgot to add in the previous tutorial about binding and connecting. I’ll take the info straight out of the ZeroMQ guide, like am doing everything else :)

To create a connection between two nodes, you use zmq_bind() in one node and zmq_connect() in the other. As a general rule of thumb, the node that does zmq_bind() is a “server”, sitting on a well-known network address, and the node which does zmq_connect() is a “client”, with unknown or arbitrary network addresses. Thus we say that we “bind a socket to an endpoint” and “connect a socket to an endpoint”, the endpoint being that well-known network address.

So with that out of the way.

In MyPull.kt we’ll have an infinite loop where we’ll be consistently pulling data from a particular socket in our case tcp://localhost:5897

Here is the code for MyPull.kt

In MyPush.kt we’ll connect to the address where our puller has bind-ed(bound) itself 😅. And we’ll create a loop that sends “Hello” to the Puller node.

The code of MyPush.kt is as follows

We can easily interchange ZMQ.PUSH with ZMQ.PUB and ZMQ.PULL with ZMQ.SUB to see the other sockets in actions. Understanding their differences, some of which I’ve highlighted above and reading them up from the guide.we can be creative and write exercises to test if they drop messages as we’ve said and see if they behave accordingly with multiple nodes connected. In our case we could quickly create a copy MyPull.kt file, rename it to MySub.kt and change the ZMQ.PULL to ZMQ.SUB and make copies MySub1.kt, MySub2.kt and run them all with one MyPub.kt file that uses ZMQ.PUB socket to send messages.

After I run the MyPull.kt and then run MyPush.kt, this is the data I get printed out in the console

For MyPush.kt

For MyPull.kt

You can find the code on github