The streaming bridges — A Kafka, RabbitMQ, MQTT, and CoAP example

The streaming bridges — A Kafka, RabbitMQ, MQTT, and CoAP example

1. Introduction

Nowadays, we can rely on different streaming systems to transmit and collect our data. Some of the systems that we use, require extremely high availability. On the other hand, for other systems, high availability can be a trade-off between having a reliable system. Sometimes getting systems slower, can allow room to increase the reliability of the data being transmitted. What is fascinating about this, is that the choices are quite endless. To stream our data we can use push or pull mechanisms for example. It is very important that we are aware of the distinction between them. A pull mechanism is one where the process is constantly looping through a channel or buffer to see what’s in there. A push mechanism sends the data to a place where a process is waiting for it. These two distinct mechanisms are exactly the main difference between using Kafka and RabbitMQ respectively. Kafka has its origins in LinkedIn in 2010 and RabbitMQ was developed on February 1, 2007, by a company of a similar name: Rabbit Technologies Ltd. What’s important to notice is that RabbitMQ implements the AMQP (Advanced Messaging Queuing Protocol). RabbitMQ is also referred to as an AMQP broker. Later on, Kafka opted to develop stream buses. These…