Power of in-house chat

One of the key things to the success of a service professional marketplace like Urbanclap is the seamless communication between customers and professionals. One of these channels is via in app chat. The use case of this arises when customers do not want to share their phone numbers with the professionals. And privacy of customers is at utmost importance at UrbanlClap.

We started out this via Pubnub, a third party out of the box cross platform chat solution, which we integrated in our apps. Even though it was easy to get started with it, after about a year of it in production, we realized that for our use cases we were bound to use its features which were not sufficient for us and hence decided to develop in-house chat. Our focus was to develop something pretty fast, highly scalable, highly flexible in terms of adding new designs and features.

We did analysis around two popular protocols for chat i.e MQTT and XMPP and finally decided to go with XMPP protocol and Ejabberd as server. The reason for this decision in mainly behind power of Ejabberd in handling scale and fault tolerance. MQTT is still new in IoT market, needs a lot of work to be done and has less community support.

Ejabberd is a distributed fault-tolerant Jabber server which is mainly written in Erlang. Few people may argue that XMPP is very heavy weight and as we are developing for apps, it may drain out phone batteries very soon, but here Ejabberd plays its role in providing XMPP usage in most optimal manner. So hence we ruled out that problem in coming future.

(Overheard somewhere underground in the House of Mogae)
Random guy: “UC chat doesn’t work great”
Chatzilla monster: “Wait till I eat you for breakfast”

As we developed in house chat we got away with disadvantages of Pubnub:

1. We have two different apps both on Android and iOS for consumers and professionals and Pubnub doesn’t support two iOS keys for same account due to which we weren’t able to deliver chat notifications on both apps.
2. With in-house chat we have flexibility to add new custom designs and chat on top of chat, which was bit difficult with Pubnub before.
3. We now have more control over analytics and find out critical events over chat to optimise it further for both consumer and professionals.

We are using mysql for persistence rather than Mnesia with Ejabberd. Mysql is chosen over Mnesia due to following reasons :

1. Ease of implementation, scale and persistence.
2. Mnesia is a CP system (consistent but not available under network partitions) not an AP system (available but not consistent under network partitions), due to which if one of node fails, it requires manual intervention.
3. Analysis on top of Mnesia and its integration with various BI tools is another area where you need to start investing from scratch.
4. Mnesia is mainly designed for Erlang systems and keeping data in-memory rather than on disk, so scalability becomes an issue as data grows.

We developed many modules around Ejabberd in erlang to make it more robust for our use cases like chat notification module which solved our previous problem. Currently we are running with fault tolerant, highly scalable in-house chat with more than 50% usage on request level. This has been huge accomplishment for us and moved our overall connects between consumer and professionals from 65% to 80%.

We monitor health of our system through grapherl which is a module over Ejabberd and captures data that we send, to find out historical connects, user online status, message delivery and user coming to our platform.

Showcasing growth in total registered users over time

Blue : current_online_users over time, Orange : current_online_professionals over time, Green : current_consumers_online

We have multiple features in place:

1. We have offline message delivery feature. If a user is offline and user is trying to chat, we provide functionality to do so and send the messages when the user comes online.
2. Delivered and read tick marks on each and every chat message
3. Ordered delivery of chat messages
4. User signs out and in, we sync all his history of messages offline without blocking him to do any further functionality

We faced multiple challenges when we were migrating our complete chat user base from Pubnub to Ejabberd :

1. Given we have 4 apps in total (2 android and 2 iOS), migrating previous chat data which we downloaded through Pubnub API to Ejabberd was really tricky.
2. Due to few product constraints, we couldn’t force update all 4 apps to new in-house chat infra and hence we came up with architecture which gave flexibility to run both Pubnub and Ejabberd running at same time. It was one of the complex and biggest challenge.

It has been really interesting project for us and quite impactful in terms of solving connects between consumers and professionals. Thanks to Karan Ahuja, Aditya Chowdhary, Amandeep Singh and Kirat Chhina for contributing on building Chatzilla.

Regards,

Mohit Agrawal