Connectivity in the palm

I did messaging platforms three times, each time using different technology. In university, I was charmed by relational algebra and keep everything in normal forms and OLAP cubes. Later I discovered the deep world of LDAP and X.500 protocols along with BER encodings and ASN.1 notation.

SyncML — Forever young

We started at Synrc in 2005 the Contact Sync startup based on SyncML protocol and its extensions. Those days all NOKIA phones were able to sync over SyncML and seemed nothing could stop us, unless only NOKIA fall. We used Funambol SyncML server and Wildfire XMPP. I had in mind business organizer based on an 8-dimensional database with Events, Messages, Projects, Locations, Objects, Dates, Contacts, Tasks where you can connect any to any. In 2010 I tried to implement SyncML server in Erlang but didn’t finish the work because of enormous underlying WBXML stuff.

XMPP — Highway to hell

The first my Erlang project was XMPP based chat messenger, I tried XMPP over BOSH, I even wrote my own Chat application for Haiku Operating System that was compatible with Facebook, Google and Jabber servers. The biggest drawback of using pure XMPP is that PubSub should be ideologically implemented as an XEP extension to be consistent with protocol federation. However the more you dive into XML the more you start getting lower realms. XMPP was a failed project. Nowadays nobody will start a messaging platform relying on XML. In 2012 Facebook let XMPP went away. However couple successful XMPP based Erlang chat implementations was in our memory: Crytek Warfare messaging bus and Facebook chat, btw Ukrainian Crytek team was on my Erlang courses two times.

AMQP — Safe in New York City

The second take was made using AMQP as PubSub and Riak as a storage. Everything was promising but later I discovered how I getting lost in a number of meaningless layers of AMQP dependencies such a rabbit_common and amqp_client and plus an additional layer of your callbacks. Bullshit using bullshit. I started critically reviewing the protocol implementation and found that it’s not well suited neither for chat messenger nor for simplicity. Riak circles of the layers also was a signal for Basho’s falling. However, Riak is still in my memory stays the simplest and operator friendly best-in-class database, not so fast though.

MQTT — Stairways to heaven

In 2012 Facebook started to use MQTT protocol for its messenger. MQTT should be very interesting to chat developers as it has such features as persistent sessions (clean_session=false) and guaranteed message delivery (qos=2) which means you can hide away these things from your application protocol and put them deeper. However usually MQTT servers provide any level of persistence only as a paid feature. Thanks to devas there are couple MQTT implementations that could be used as a good start. I took EMQ by Feng Lee and extended it with KVS persistence. Retain topics are completely useless when you think it at scale, but all other goodies from MQTT are very handy, it keeps your basic chat and presence server under 2 000 LOC.

EMQ — Persistence

Internally EMQ subscription system consists of two tables: mqtt_subscription (client key), mqtt_subscriber (topic key). mqtt_subproperty is used for storing topic properties such as qos. mqtt_topic is mqtt_route are used for node routing. Out of the box, EMQ is storing everything in ets, and between restarts subscriptions are gone. So we modified EMQ to store everything and mnesia disc_copies, while indexing mechanism used for ets are still used as disc_copies stores everything in ets as a cache. The other persistence you need to think about is a inflight messages used in qos=2 configuration which enables (MQTT 3.1.1) persistent offline messages. If you need really persistent messages between server restarts you need to substitute inflight module for mnesia too.

N2O — RPC over MQ

For that project, I took N2O Application Server and modify it to work not over cowboy, but over emqttd. The things differ as in WebSocket your call live in one request/reply sync context, but in MQ you need to use request and reply topics. This pattern sometimes called RPC. For requests, N2O uses events topic and for responses actions topic. N2O spawns a ring of request performers on all cores. That workers handle and run BERT protocol messages from MQTT clients. The workers are clients too. This is all of the N2O usage here.

MacBERT — Swift code from Erlang types

Binary Erlang Terms are a bytecode format of erlang structures. We generate Swift parsers from HRL record files directly, and not using other descriptive formats. We have protocol sums in generated specifications, so you can handle complex subprotocols.

-record(io, { code=[] :: [] | #ok{} | #error{},
data=[] :: [] | <<>> |
{ atom(), binary() | integer() } }).

Chain(types: [
Model(value:Tuple(name: "io", body: [
Model(value:Chain(types: [
Model(value: Tuple(name:"ok", body: [
Model(value:Atom())])),
Model(value: Tuple(name:"error", body: [
Model(value:Atom())]))])),
Model(value:Tuple(name:"", body:[
Model(value:Atom()),
Model(value:Chain(types: [
Model(value:Binary()),
Model(value:Number())]))]))])) ])