Scaling WebSockets in Spring services

Photo by Carl Nenzen Loven on Unsplash

Let’s imagine that we have a simple chat application in which the frontend communicates with the backend via rest and WebSockets used for chatting. We realize that one instance of the application starts not to cope with the load.

It’s not a trivial task to scale microservices that uses WebSockets. With a simple launch of another instance under a default round-robin load balancer, we may get a situation when one user connects to instance-A and the second user connects to instance-B. And now, our backends must somehow understand where to send incoming messages.

Custom Load Balancer

The first option that may come to mind is to write an intelligent load balancer that will redirect users from the same chat to the same instance.

Several problems can arise here:

1. If users communicate with many people simultaneously, then for each chat, you need to open a new WebSocket connection.
2. If there are too many people in the chat, then one backend instance may not cope. For a chat specifically, this is unlikely, but the chat example is an oversimplification. In real life, this problem is not uncommon.

Message Broker

So let’s take a different path. Fortunately, in addition to the in-memory broker for WebSockets, Spring also has a BrokerRelay that delegates the processing of queues to a third-party broker.

Now is the time to choose a broker. There are many options, and we will not consider everything. Of the most popular:

1. Apache Kafka is poorly suited because it is not designed for a large number of dynamically generated queues.
2. Redis PUB/SUB handles this load best, but you can’t get it out of the box for Spring.
3. Other options are RabbitMQ and ActiveMQ.

Setting up broker relay

Routes naming

Route naming could be a problem if we chose RabbitMQ. If we use the standard slash paths, then we can see the messages:

The point is that RabbitMQ does not allow slashes after standard routes. Thus, if we send a message to the routes /topic/ or /queue/, then there should be no other slashes in the name. The easy way here is to rename all destinations on the frontend and backend sides by replacing the slashes with dots.

The situation becomes more complicated if the frontend (or another application connecting via WebSockets) has a different release cycle. Or you need to maintain compatibility with other versions. In this case, you can write an Interceptor that will replace the destination in messages.

But you should remember that Spring saves subscriptions before sending them to the MessageChannel, so you will have to replace destinations at the stage of sending to the broker itself and receiving from it. You can see the entire communication schema in the Spring documentation.

An alternative solution here is to use a different broker. For example, there is no such problem in ActiveMQ.

SubscribeMapping

If your application uses the @SubscribeMapping annotation in controllers for routes delegated to an external broker (/topic/, /queue/), this will also be a problem.

When Spring receives a message with such destinations, it saves the subscription and then sends the message to the broker, bypassing the controllers. Thus, the method annotated with @SubscribeMapping will never be called.

There are two solutions here. The first solution doesn’t require changing the business logic, while the seconds one does:

1. You could add a BeanPostProcessor that will scan all methods with this annotation and invoke them on the SessionSubscribeEvent event. The main problem here is that the SessionSubscribeEvent is raised before the subscription message is sent to the broker. Therefore, a situation is possible when we call a method that sends something to a topic, but the user himself has not yet subscribed to this topic. The solution to this problem is tricky and requires waiting for the dispatch event through Interceptors.
2. You can replace @SubscribeMapping in controllers with the @GetMapping REST annotation. Thus, the initial state will be obtained not by WebSockets but by REST. Such a solution will also require changes on the frontend side.

Conclusion

Scaling microservices that use WebSockets could be more complicated if you didn’t consider it when building your application. The primary solution comes down to using an external message broker. When you connected the message broker for the first time, the application may not work correctly. But there are only two main problems (route naming, @SubscribeMapping), and both have a solution.