SSEGWSW: Server-Sent Events Gateway by Service Workers

Published in

IT’s Tinkoff

7 min readMar 27, 2020

Hello!

My name is Alexandr and I work as an architect at Tinkoff Business.

In this article, I want to talk about how to overcome the restriction of browsers on the number of open long-lived HTTP connections within the same domain using a service worker.

If you want, feel free to skip the backstory, description of the problem, search for a solution. Just immediately proceed to the result.

Backstory

Once upon a time, there was a chat in Tinkoff Business that worked on WebSocket.

It ceased to fit into the design of the account dashboard after some time, and in general, it needed a rewrite from angular 1.6 to angular 2+. I decided that it was time to start updating it. The backend colleague found out that the chat front-end will change and he suggested at the same time redoing the API, in particular — changing the transport from WebSocket to SSE (server-sent events). He suggested this because all connections were broken when updating the NGINX config. And restoring them is difficult.

We discussed the architecture of the new solution and came to the conclusion that we will receive and send data using ordinary HTTP requests. For example, to send a POST: /api/send-message message, to get a list of dialogs GET: /api/conversations-list, and so on. And asynchronous events like “a new message from the interlocutor” will be sent via SSE. So we will increase the fault tolerance of the application: if the SSE connection will fall off, the chat still works, only it doesn’t receive realtime notifications.

Besides chat, we also used a WebSocket for the “thin notifications” component. This component allows you to send various notifications to the user’s personal account. For example that the import of accounts (which mays take several minutes) has been completed successfully. We moved this component to a separate SSE connection to completely abandon WebSocket.

Problem

When you open one browser tab, two SSE connections are created: one for a chat and one for thin notifications. Ok, let them be created. Do we feel greedy? We are not greedy, but browsers are greedy! They have a limit on the number of concurrent persistent connections for a domain. Guess how much is in Chrome? Right, six! I opened three tabs — I filled the entire connection pool and I can no longer make HTTP requests. This is true for the HTTP / 1.x protocol. In HTTP / 2 there is no such problem due to multiplexing.

There are a few ways to solve this problem at the infrastructure level:

Both of these methods seemed expensive since a lot of infrastructures would have to be affected.

So first we tried to solve the problem on the browser side. The first idea was to make some kind of transport between the tabs, for example, through the LocalStorage or Broadcast Channel API.

The meaning is this: we open SSE connections in only one tab and send the data to the rest. This solution also did not look optimal, since it would require the release of all 50 SPA, which make up the Tinkoff Business personal account. Releasing 50 applications is also expensive, so I continued to look for other ways.

Solution

I recently worked with service workers and thought: is it possible to apply them in this situation?

To answer this question, you first need to understand what service workers can do in general. They can proxy requests, it looks something like this:

self.addEventListener('fetch', event => {
    const response = self.caches.open('example')
    .then(caches => caches.match(event.request))
    .then(response => response || fetch(event.request));

    event.respondWith(response);
});

We listen to events for HTTP requests and we respond as we like. In this case, we are trying to respond from the cache, and if it does not work out, then we make a request to the server.

Ok, let’s try to intercept the SSE connection and answer it:

self.addEventListener('fetch', event => {
    const {headers} = event.request;
    const isSSERequest = headers.get('Accept') === 'text/event-stream';

    if (!isSSERequest) {
    return;
    }

    event.respondWith(new Response('Hello!'));
});

In-network requests we see this picture:

And in the console this:

Already not bad. The request was intercepted, but the SSE does not want a response in the form of text/plain but wants text/event-stream. How to create a stream now? Maybe I can respond with a stream from a service worker? Well let’s see:

Excellent! The Response class takes ReadableStream as a body. After reading the documentation, you can find out that ReadableStream has a controller that has an enqueue() method — with its help you can stream data. Suitable, I take it!

self.addEventListener('fetch', event => {
    const {headers} = event.request;
    const isSSERequest = headers.get('Accept') === 'text/event-stream';

    if (!isSSERequest) {
    return;
    }

    const responseText = 'Hello!';
    const responseData = Uint8Array.from(responseText, x => x.charCodeAt(0));
    const stream = new ReadableStream({start: controller => controller.enqueue(responseData)});
    const response = new Response(stream);

    event.respondWith(response);
});

There is no error, the connection hangs in pending status and data does not arrive on the client-side. Comparing my request with a real server request, I realized that the cause is in the headers of the answer. Those headers must be specified for SSE requests:

const sseHeaders = {
    'content-type': 'text/event-stream',
    'Transfer-Encoding': 'chunked',
    'Connection': 'keep-alive',
};

When you will add these headers, the connection opens successfully, but the data is not received on the client-side. This is obvious because you can’t just send a random text — there must be some format.

You need to send data from the server with a specific data format, and it is explained well at javascript.info. It can be easily described with one function:

const sseChunkData = (data: string, event?: string, retry?: number, id?: number): string =>
    Object.entries({event, id, data, retry})
    .filter(([, value]) => ![undefined, null].includes(value))
    .map(([key, value]) => `${key}: ${value}`)
    .join('\n') + '\n\n';

The server must send messages separated by a double line break \n\n to comply with the SSE format.

The message includes these fields:

data — message body, several data in a row are interpreted as one message, separated by line breaks \n;
id — updates the lastEventId property that is sent in the Last-Event-ID header when reconnecting;
retry — the recommended delay before reconnecting in milliseconds, cannot be set using JavaScript;
event — the name of the user event, indicated before data.

Let’s add the necessary headers, change the answer to the desired format and see what happens:

self.addEventListener('fetch', event => {
    const {headers} = event.request;
    const isSSERequest = headers.get('Accept') === 'text/event-stream';

    if (!isSSERequest) {
    return;
    }

    const sseChunkData = (data, event, retry, id) =>
    Object.entries({event, id, data, retry})
    .filter(([, value]) => ![undefined, null].includes(value))
    .map(([key, value]) => `${key}: ${value}`)
    .join('\n') + '\n\n';

    const sseHeaders = {
    'content-type': 'text/event-stream',
    'Transfer-Encoding': 'chunked',
    'Connection': 'keep-alive',
    };

    const responseText = sseChunkData('Hello!');
    const responseData = Uint8Array.from(responseText, x => x.charCodeAt(0));
    const stream = new ReadableStream({start: controller => controller.enqueue(responseData)});
    const response = new Response(stream, {headers: sseHeaders});

    event.respondWith(response);
});

Oh, my Glob! Yes, I made an SSE connection without a server!

Result

Now we can successfully intercept the SSE request and respond to it without going beyond the browser.

Initially, the idea was to establish a connection with the server, but only one, and to send data to tabs from it. Let’s do it!

self.addEventListener('fetch', event => {
    const {headers, url} = event.request;
    const isSSERequest = headers.get('Accept') === 'text/event-stream';

    // We process only SSE connections
    if (!isSSERequest) {
    return;
    }

    // Response Headers for SSE
    const sseHeaders = {
    'content-type': 'text/event-stream',
    'Transfer-Encoding': 'chunked',
    'Connection': 'keep-alive',
    };
    // Function formatting data for SSE
    const sseChunkData = (data, event, retry, id) =>
    Object.entries({event, id, data, retry})
    .filter(([, value]) => ![undefined, null].includes(value))
    .map(([key, value]) => `${key}: ${value}`)
    .join('\n') + '\n\n';
    // Table with server connections, where key is url, value is EventSource
    const serverConnections = {};
    // For each url, we open only one connection to the server and use it for subsequent requests
    const getServerConnection = url => {
    if (!serverConnections[url]) serverConnections[url] = new EventSource(url);

    return serverConnections[url];
    };
    // When we receive a message from the server, we forward it to the browser
    const onServerMessage = (controller, {data, type, retry, lastEventId}) => {
    const responseText = sseChunkData(data, type, retry, lastEventId);
    const responseData = Uint8Array.from(responseText, x => x.charCodeAt(0));
    controller.enqueue(responseData);
    };
    const stream = new ReadableStream({
    start: controller => getServerConnection(url).onmessage = onServerMessage.bind(null, controller)
    });
    const response = new Response(stream, {headers: sseHeaders});

    event.respondWith(response);
});

The same code on GitHub.

I got a pretty simple solution for a not-so-trivial task. But, of course, there are still many nuances. For example, you need to close the connection to the server when closing all the tabs, fully support the SSE protocol, and so on.

We have successfully solved all this. I’m sure it will not be difficult for you too!

SSEGWSW: Server-Sent Events Gateway by Service Workers

Backstory

Problem

Solution

Result

Written by Alexandr Kazachenko