Python and Server-sent Event

I have had the chance to work on an IoT project that used HTML5 Server-sent Event(SSE) in Python and here is why I thought Python is a great server-side language for SSE.

Server-sent Event (SSE), unlike the sexier sister like the websocket, is a unidirectional “push.” What this means is it is ideal for features that do not required bidirectional real-time updates between the server and client such as device monitoring or social updates.

What’s great about SSE is it is very easy to implement because it’s just a simple HTTP request (unlike websocket which is a new protocol). Instead of terminating after the server gets a request, the connection between the server and client is kept open for the server to push data changes to the latter.

Short Intro To Generators

Python generator is actually a new pathway for Python to enter concurrency, and it’s being implemented under the hood by many libraries of such nature.

A generator function, unlike a regular function, does not terminate upon a return statement. Therefore, it can retain states inside it until the internal loop is exhausted.

Here is a simple function.

def myfunction(x, y):
    for i in xrange(10):
        print x + y
        x += 1

Since print statement just prints to the stdout, the function continue to increment x until the loop finishes.

But if, instead of printing, we want to retrieve that value instead? There is no way of doing so in a normal function because a return statement would always break out of the function, and the local variable x is thrown away. This is where generator functions shine.

def myfunction(x, y):
    for i in xrange(10):
        yield x + y
        x += 1

This generator function yield the value captured in each iteration without breaking out of the function. Actually, the function “return” a generator object, which is a iterable type (i.e. can be looped ever).

for n in myfunction(3, 2):
    print n

This is where the concurrency magic happens. In every interation, the interpreter will jump back to the generator function, get the latest state of the stored value of x, and when it hits yield, it jumps right back and print n out to stdout.

Server-sent Event (SSE)

SSE is a native HTML5 feature that allows the server to keep the HTTP connection open and push data changes to the client. This is actually beneficial client-side because the event/data sent from the server can actually be captured by client-side Javascript.

// Client-side Javascript in the HTML
var targetContainer = document.getElementById("this-div");
var eventSource = new EventSource("/stream");
eventSource.onmessage = function(e) {
    targetContainer.innerHTML = e.data;
};

Here, the targetContainer “consume” data from the EventSource object, which is instantiated by a URI route or a file path.

The targetContainer DOM will change to when ever the server push a new server-sent event to the client. That simple.

Generator + SSE

So why are Python generators good with SSE? It’s simply because they can keeping looping and yielding data and handing it to the client very seamlessly. Here is a simple Python implementation of SSE in Flask:

@route("/stream")
def stream():
    def eventStream():
        while True:
            # Poll data from the database
            # and see if there's a new message
            if len(messages) > len(previous_messages):
                yield "data: 
                    {}\n\n".format(messages[len(messages)-1)])"
    
    return Response(eventStream(), mimetype="text/event-stream")

This is a simple hypothetical event source that checks if there’s a new inbox message and yield the new message. For the browser to acknowledge a server-sent message, you’ll have to comply to this format:

"data: <any_data>\n\n"

You have the option to also send with the data the event and id.

"id: <any_id>\nevent: <any_message>\ndata: <any_data>\n\n"

Note that the fields do not have to be in any order as long as there is a newline (\n) for each field and two (\n\n) at the end of them. With additional event field, you can have more control how you push data to the browser.

// Client-side Javascript in the HTML
var targetContainer = document.getElementById("this-div");
var eventSource = new EventSource("/stream");
eventSource.addEventListener = (<any_message>, function(e) {
    targetContainer.innerHTML = e.data;
    
    if (e.data > 20) {
        targetContainer.style.color = "red";
    }
};

This will basically render the DOM with the latest data on the specified event message and change the color to “red” when it exceeds 20.

Conclusion

Server-sent Streaming is really ideal for server-push notifications, device monitoring and all other tasks that do not require real-time push back from the client. There are ways to actually implement the bidirectional, websocket-style communication by using pub-sub model, likely with the help from Redis.