Server Sent Events (SSE) Streams with Node and Koa

Photo by Mathew Schwartz on Unsplash

When you want to have real time updates on a web application, you can rely on old-fashioned periodic polling or try to use some push capable modern technology. Your first impulse may be using WebSockets. However, if you just want to receive data from the server you can use Server Sent Events.

Traditionally, a web page has to send a request to the server to receive new data; that is, the page requests data from the server. With server-sent events, it’s possible for a server to send new data to a web page at any time, by pushing messages to the web page. These incoming messages can be treated as Events + data inside the web page.

You can check this post to see the difference between SSE and Websocket and make your own opinion of when to use one or the other. For my use case, receiving updates from the server periodically, I will stick to SSE.

SSE basics with Koa

Let’s start by building a Koa based HTTP server.

• It will have a catch all route that responds with a 200 status.
• It will have a /sse endpoint that, upon receiving a request, will make sure our connection stays open by adjusting some socket params and return the appropriate HTTP headers to start a new SSE stream.
• We will create a new PassThrough stream (a stream that simply passes the input bytes across to the output) and pass it as our response body.
• Lastly we will generate a data feed using a simple interval that will periodically write the current timestamp to the stream; therefore, pushing that data across the open connection to the client.

Notice two things: first, the output data must conform to the SSE format. Second it’s mandatory to return a stream as the body response to ensure that Koa does not close the connection. You can dig in the Koa source code (check this method) to see how Koa handles the response. If you take a look you’ll see that Koa will send the body content to the HTTP response stream unless you use another stream as the body. In that case it will pipe the streams; therefore, the response stream won’t be closed until we close our PassThrough stream.

To test our SSE stream in the browser we should use the EventSource API.

Visit http://localhost:8080 in your browser, open a console and drop the following snippet to consume the server messages.

Closing the stream

If you reload your browser or close your source (using the close() method) your server will break. The interval will try to write on the stream, and… it’s gone!

We have to be careful when dealing with closures, in this case our stream. Our interval does not know that it has to stop feeding data to the stream.

To solve it, we can attach ourselves to the stream close event and “unsubscribe” from the data feed.

Broadcasting data

The previous examples generate a new data feed for each connected client. In real world scenarios we also want to broadcast the same data to different clients.

We can add a simple EventEmitter and move the data generation outside of the connection code to see how this could work.

Formatting the stream data

As I mentioned before the SSE stream data must conform to a standarized format. To ease the pain of translating from data objects to SSE messages we will swap our PassThrough stream with a custom stream Transformer.

• The transformer (line 5) will convert objects into SSE text format.To simplify the example, we will just handle data only messages (line 13).
• Our data feed will emit a data object with a timestamp key (line 22).
• Our event listeners will write the raw data to the transformer (line 46).

We also have to make a minor adjustment to our client to process the new JSON formatted data.

Summing up

This is just a barebones example to illustrate the SSE technology: how easy is to use it in Node and how you have to use streams to build SSE into a Koa service. You can extend the example’s SSEStream to support all the Event Stream format and much more.

As I mentioned at the beginning of this story, when dealing with server to client unidirectional communication you can choose between websockets and Server Sent Events. Each one has pros and cons. Learn both of them and choose the one that fits your problem.