Server Sent Events

Published in

Tokopedia Engineering

5 min readJul 21, 2020

Tokopedia, Server Sent Events reducing upload time — SSE @ Tokopedia

Introduction

At Tokopedia, there is always a challenge and opportunity to improve the experience of sellers or merchants in adding /editing products. Some other platforms support single add/edit flow, and the platform of Tokopedia extends to one-click upload of multiple products using predefined excel or CSV format. This article explains how to build a progress bar API using SSE (Server Sent Events) for these file uploads.

The pros and cons of SSE have been discussed in length in the articles mentioned in the reference. This article is aimed to describe how to achieve it using SSE, and problems that one may encounter during the setting up the production environment, especially when there are multiple proxies involved in between.

Typical File Upload Process

A typical process of file upload on any platform is

Upload File Progress Bar High Level Flow

Though there are some common ways such as Polling or Websockets to Show Progress of Uploaded file, we implemented SSE to achieve the Progress Bar for file upload on the client end.

High-level Architecture

High Level Upload file and Progress Bar API Server Sent Event Architecture

How to make Go handler ready for SSE?

Let us say /progress is the url of the service mapped to Handler ProgressHandler, and the service runs on 1991 port.

All you need to do is set below headers in the handler to the server the client.

func ProgressHandler(w http.ResponseWriter, r *http.Request)(interface{}, error) {
 w.Header().Set(“Connection”, “keep-alive”)
 w.Header().Set(“Content-Type”, “text/event-stream”)
 w.Header().Set(“Cache-Control”, “no-cache”)
 w.Header().Set(“X-Accel-Buffering”, “no”):
:}

Cache-Control: no-cache

This header tells the client not to store any data in the local cache, and always read the data sent by the server.

Content-Type: text/event-stream

This is the key to all the communications that happens between client and server in SSE.

Tuning Nginx

location /progress {
        proxy_set_header Connection '';
        proxy_http_version 1.1;
        proxy_cache off;

        proxy_pass http://localhost:1991;
    }

proxy_set_header Connection ‘’;

TCP Connections in HTTP1.1 are persistent by default, so keeping Connection header empty ensures that it is not overwritten by another value such as “close”. Refer to this RFC for more detail.

proxy_http_version 1.1;

From ProgressHandler, the “Connection” header is set as “Keep-alive”. This header tells the client to maintain a persistent connection. proxy_http_vesion 1.1 ensures that “Connection: Keep-alive” header is passed back to upstream.

In HTTP 2.0 the request and response are interleaved and head-of-line blocking does not happen. This improvement of HTTP2.0 does solve the asset rendering problem, but since the TCP connection has to be established only once, HTTP2.0 does not support keep-alive headers or rather filters this header.

Today, most of the SLBs and proxies are default to HTTP2.0. At Nginx proxy layer proxy_http_vesion 1.1 ensures that the request adheres to HTTP1.1 conventions. If SLB is configured with HTTP2.0, as of today there is no way to configure the same SLB to support both HTTP versions, so better to create a new SLB configured with HTTP1.1 and channel the request that requires streaming through this SLB.

proxy_cache off;

This is needed to avoid any kind of caching. Some blogs do suggest that this is not mandatory, but based on our findings, this is required to ensure that content buffer is passed to the client without any caching.

Don’t add these modules in Streaming API location block

chunked_transfer_encoding off;

In SSE, content-length is never set in response headers because there is no clarity on data length. By default if Content-Length is not set in HTTP Response Headers most HTTP servers will set Transfer-Encoding: chunked; . This theory may lead to the conclusion that chunking must be turned off to ensure the real-time nature of SSE. However, this is not the case and hence this Nginx module is not needed. Some blogs may suggest adding this for the hit and trial method.

proxy_buffering off;

Considering the real-time nature of SSE, and that there is the streaming of content between client and server, we need to turn off buffering. However, adding “X-Accel-Buffering”, “no” in HTTP handler response header ensures that buffering is off specifically for the API serving SSE, and thus this module is not needed in the location block.

Unknown Territory

In our case, initially, the SLB was configured with HTTP2.0, and we didn’t get a continuous stream of response. However, there was always a response on the client after 32kb of data was ready in the stream. This chunked buffering was defeating the purpose of the stream API.

We still need to dive deeper and find out how we can make SSE work on HTTP2.0.

Server Sent Event and Server Push Event

These are two extremely different things. Server Sent Events are long-running unidirectional TCP connections. These are used to send data from server to client as explained in detail above.

Server Push Event is purely for assets and contents and is used to send assets to the client before the assets are asked for. This a way to bid goodbye to all the inlining, asset compression, and image spriting.