Centrifugo 2 plans and new real-time messaging library for Go
Last year I left Mail.Ru where I had been working for almost 7 years. I joined Avito in the end of summer — the biggest Russian classified where I am now working on internal messenger platform. If you’ve read some of my previous posts then you know that messaging, especially real-time messaging is something that makes me most excited.
At new job I met new interesting technologies I never worked with before — Kubernetes, RabbitMQ, MongoDB. Changes in life, new experience and new team inspired me to continue work on Centrifugo project and I am now happy to announce some interesting things.
First of all, I’ve broken it. Last several months I was working on Centrifugo 2 which is backwards incompatible with 1.x. And this fact actually scares me. I did my best during last several years developing Centrifugo ecosystem for various languages — API and client libraries for Android, iOS, Python, NodeJS, Ruby, PHP, Go. Some of those libraries I wrote myself, some with great help of open-source community. Now it looks like I should start from scratch. And this will be hard.
Let me justify why I made this decision and making version 2 backwards incompatible. Internal Centrifugo client-server protocol was born many years ago when Centrifugo was written in Python. Of course protocol changed a bit during its life cycle but was pretty stable in its core. One of the biggest features in Centrifugo 2 will be binary websocket support. And as Centrifugo has its own JSON framing on top of Websocket/SockJS connection for various messages (connect and authenticate, subscribe on channel, messages of different types received from channel etc) so I need extra encoding layer on top of binary Websocket frames and the choice was pretty obvious — Protobuf. But I quickly realized that something that is possible with JSON is not a good fit for Protobuf. Internal messages must be really well organised — and it’s actually a good thing that Protobuf enforces this — so I had to make some backwards incompatible changes into Centrifugo protocol. Actually without pretty huge refactoring it would also be almost impossible to implement some things I’ll tell below.
But! Centrifugo server will solve the same task as before and will have the same concepts as before. It’s a standalone server that works near your application backend, helps with persistent connections and has API to deliver real-time messages to connected clients. The work on version 2 is currently concentrated in c2 branch.
Here I want to highlight some v2 features in short without diving deep in details. There are actually much more then I write here but let’s talk about the most important parts.
As I already pointed above internal protocol is now fully described by Protobuf schema. Centrifugo 2 can support not only JSON for client-server communication but also binary Websocket frames. Making it work with two encoding formats was quiet challenging but looks like it already works in Centrifugo v2 development branch.
The next thing to mention is that Centrifugo now supports GRPC bidirectional streaming for client-server communication. This is pretty experimental at moment but I think it’s a great feature — to have a possibility to work over Websocket with browser clients and over GRPC with mobile app clients for example.
I was always sure that Centrifugo has some useful code inside that can be reused even if server itself doesn’t fit someone’s needs or expectations. Some people asked me if it’s possible to use Centrifugo as library from Go code. The answer was:
well, yeah, you actually can. But keep in mind that Centrifugo is a standalone server and it’s internal code designed for the needs of that server and can be changed at any moment
One of my goals during developing Centrifugo 2 was segregate real-time core of Centrifugo into separate library. While I still have not finished work you can find a work in progress repo with that library at https://github.com/centrifugal/centrifuge. Centrifugo 2 will use this library internally. And it’s now possible for Go programmers to use it for building real-time Go applications.
Let’s look at simple example of using Centrifuge library in Go application:
And on client side, for example in browser:
This example demonstrates how to use Centrifuge library for simple things like logging when clients connect to server and disconnect from it and also responding to RPC requests from clients. Everything works over Websocket in this example. But it’s also possible to start SockJS handler or run GRPC server to do the same over alternative transports. And you can start many Centrifuge nodes and connect them via Redis PUB/SUB and with sharding support. Have presence, history, join/leave events and message recovery features out of the box — i.e. everything Centrifugo server had will be in library. As library has binary websocket support one can even create real-time multiplayer games with it.
It’s also possible to wrap Websocket handler with any middleware and authenticate/authorize connections and actions they want to do using any mechanism you would usually use in Go application.
The bad news is that we don’t have clients yet (even browser one though minimal working prototype is in centrifuge-js c2 branch) — so all things I described above are mostly on paper at moment even though server part is mostly finished. I hope situation with clients will change with time and I will be able to show more soon.
So… This is a work in progress at moment and many things to do are still in roadmap. I am sharing this post to ask for open-source community help and feedback. Are you personally interested in library like this? Can you help with library API design or code review? Could you try to build real-time application with it and provide a feedback? Maybe you are interested in writing client? Any help is appreciated.
If you want to help with project in some way you can fire a message to me in Centrifugo Gitter chat or via email — see my Github profile.
