A publisher-subscriber connectivity solution using the Hype SDK
1. The publisher-subscriber paradigm
We currently live in a period of human history in which evolution is being driven by accessibility to all kinds of information, which can be quickly retrieved and is widely disseminated. This easiness to communicate and propagate information is mainly possible due to the evolution of computer networks, which ultimately led to the creation of the Internet.
In traditional computer networks, when two parties want to communicate they establish a connection and communicate directly. This means that the parties are tightly coupled, and if any of them fails the communication is no longer possible.
The publisher-subscriber communication paradigm uses a different approach. In this paradigm there are publishing parties, who publish channel-specific messages, and subscribing parties, who subscribe channels in which they have interest. In this paradigm, the coupling between publishers and subscribers is very loose. When a party publishes information to a channel it does not need to know who subscribed it, and when a party subscribes a channel it does not need to know who the publishers are. This way, the publishers and the subscribers are completely independent from each other. For this to happen, an intermediate Broker is responsible for keeping track of the parties that subscribed each channel and forward relevant messages to them. Thus, when a party subscribes a channel, it sends a message to the Broker signaling its intention, and when publishing to that channel, it sends a different message to the Broker, which forwards it to the current subscribers
2. Traditional publisher-subscriber system overview
In a traditional publisher-subscriber system, a Broker mediates the involved parties in order to manage channels and respective subscriptions. Figure 1 illustrates an example of a traditional system composed by 3 parties: Bob, Alice, and Eve, plus a Broker. In this case, both Alice and Eve subscribe the Weather channel by sending a subscription request to the Broker, which in turn updates its subscription list.
Assume that Bob publishes a message to the Weather channel, by sending it to the Broker. This is illustrated in Figure 2. Notice that Bob has no idea that Alice or Eve had previously subscribed the channel. In fact, Bob does not even know if anyone has subscribed the channel at all, meaning that the entities are loosely coupled. Once the Broker receives the message to be published on the Weather channel, it propagates it to the interested parties according to its Weather channel subscriptions list, thus forwarding the message to Alice and Eve.
Unsubscribing channels is also possible, as is shown in Figure 3. In this example, Eve unsubscribes the Weather channel to stop receiving weather related messages. To do this, Eve sends an unsubscribe request to the Broker which in turn updates its subscription list. Were Bob to publish a second message at this point and Eve would not receive it, while not interfering with Alice’s subscription in any way.
The traditional publisher-subscriber system described above suffers from a severe disadvantage: a single point of failure. A single point of failure is a component of a system that, by failing, leads to the collapse of the entire system. In the traditional publisher-subscriber model, the single point of failure is the Broker. If the Broker fails, the messages will not be delivered, since the Broker serves as intermediary between the publishers and the subscribers. This disadvantage can be overcome by creating a publisher-subscriber paradigm powered by the Hype SDK.
3. Hype SDK
The Hype SDK is a cross-platform mesh networking technology, created with the intent of improving connectivity on all kinds of devices. It works by connecting them together, forming a local network, using existing connectivity technologies, such as Bluetooth or Wi-Fi. This network enables content to hop between the devices, until it reaches a destination or an Internet exit point, improving range and deliverability. The content is protected with end-to-end encryption, meaning that only those participating on a conversation are capable of decrypting what is being exchanged. This paradigm eliminates the need for infrastructure, while at the same time taking advantage of it when available.
The Hype SDK enables the creation of a decentralized peer-to-peer publisher-subscriber system that does not require a centralized Broker to serve as an intermediary. By taking advantage of the decentralized properties of mesh networks, the single point of failure is no longer an issue. The proposed solution works as follows:
- Every peer in the network can subscribe a channel or publish in one;
- Each peer and each channel have a unique identifier;
- The subscription management is made in a decentralized way by the peers participating in the mesh network;
- Each channel is managed by the peer of the system which has the closest identifier to the channel identifier;
- If the peer responsible for managing a channel fails, the system can adjust and choose another peer to manage it, therefore eliminating the single point of failure issue;
- At each time, every peer in the network can infer the peer responsible for managing a given channel so it can send subscribe, unsubscribe or publish requests to that peer. That peer acts as a Broker for that specific channel.
By using the Hype SDK it’s possible to create a powerful mesh network that serves as the foundation for a decentralized publisher-subscriber system. And the best of it all is that there’s no need to worry about network issues, since the Hype SDK deals with them automatically!
4. A Publisher-Subscriber system based on the Hype SDK
The Hype SDK creates and manages mesh networks, one such network being illustrated in Figure 4. This network manages 4 peers: Bob, Alice, Eve and John, each having a unique identifier. This basic setup serves the purpose of illustrating how the publish-subscribe paradigm can be applied in a mesh, although such networks can grow extensively and become much more complex
Channels are managed according to rule number 4 above: each channel is managed by the peer of the system which has the closest identifier to the channel identifier. Assuming the IDs as illustrated in Figure 5, Alice would be the peer managing the Weather channel, since the IDs match and is, therefore, the closest. Both Eve and John subscribe the channel by notifying Alice of their intent.
If another peer, say, Bob, was to publish a message to the Weather channel, it would do so by sending an update to Alice. This is the process shown in Figure 6. As in the traditional publisher-subscribers systems, Bob has no idea that Eve or John had previously subscribed the channel. Once Alice receives the message to be published on the Weather channel, it propagates it to the interested peers according to its Weather channel subscriptions list.
The difference between the two paradigms occurs when the managing peer goes offline. As shown in Figure 7, the peer responsible for managing the Weather channel, Alice, fails and is no longer participating in the network. The Hype SDK informs John, Bob, and Eve that Alice is no longer available, and they automatically adjust accordingly. The network automatically assigns the identifier that is the second closest to the channel’s, making sure that all peers converge on the same Broker again. In this case, Eve is chosen with ID 2. John re-sends a Weather channel subscription request to Eve. Eve adds itself to the list of subscribers. This change to the paradigm ensures that the channel keeps functioning, even if its Broker goes offline.
5. Conclusion
It’s possible to conclude that the publisher-subscriber paradigm is a good approach to be used in scenarios where it’s desired to decouple senders from recipients. This approach allows the isolation of these parties, enabling them to operate independently of each other. It can improve scalability, due to the fact that publishers and subscribers do not need to be aware of each other. This type of systems can even be expanded to allow a decoupling from a temporal perspective, enabling the recipients to receive information that was published when they were not connected.
The traditional publisher-subscriber models operate using an intermediary Broker that is responsible for managing the subscriptions and which ends up being a single point of failure. Using the Hype SDK cross-platform mesh networking technology, it is possible to create a decentralized peer-to-peer publisher-subscriber system which is more robust than the traditional, centralized, approach.
See a working demo of a publisher-subscriber system using the Hype SDK here. Source code is also available on GitHub for Android, iOS, and Linux. This system is merely one example of the use cases in which the Hype SDK can be applied to solve connectivity issues. If you’d like to explore more please visit HypeLabs.
6. Links
You can see a demonstration of a demo publisher-subscriber system powered by the Hype SDK:
