How to serve HTTP/2 using Python

I’ve released a new version of this article in August 2019, it explains how to serve HTTP/1, HTTP/2 and HTTP/3. (Please click the link to visit it).

This article was updated in late 2018 to change Gunicorn references to Hypercorn (Quart > 0.5).
The article was updated in early 2019 to update the SSL setup (Quart > 0.7).

HTTP/2, the latest version of the Hyper Text Transfer Protocol was published in 2015. It is at a high level compatible with HTTP/1 insofar as there are still requests to a method plus path and responses with a status code. Furthermore both requests and responses are accompanied with headers. At a lower level it is very different to HTTP/1, being binary rather than textual, introducing multiplexing, header compression and server-push among others.

The changes introduced by HTTP/2 primarily improve performance, especially page load times. Simply reasoned, a browser can now multiplex and pipeline multiple requests over a single connection rather than waiting for each connection in turn. In fact the browser may receive responses to requests it has yet to make via server-push.

This article aims to demonstrate the above mentioned HTTP/2 features by serving a simple website, consisting of html page and the associated css and js, via Python. Ideally the server will make use of HTTP/2 server-push to push the associated css and js to the browser, but at minimum it should pipeline the css and js requests.

HTTP/2 has pipelining, so you get a picture of pipes, although with multiplexing maybe there should only be the one pipe?

The simplest way to serve HTTP/2 is to use a reverse proxy (e.g. Nginx) to switch the protocol from HTTP/2 to HTTP/1 between the client and the Python server. This has some of the advantages to the client, e.g. multiplexing, however it cannot gain the full performance advantage and does not show how to serve HTTP/2 using Python.

The easiest way to serve a simple website using Python is to use a pre-existing web framework, however if you’d prefer to directly serve HTTP/2 you should consider the excellent hyper-h2 library and this example. It is also worth noting that almost all Python HTTP/2 implementations, including the frameworks given in this article, use hyper-h2.

Current framework state

As of late 2017 there are two Python frameworks that directly support HTTP/2, namely Twisted and Quart with only the latter supporting server-push (disclaimer I work on Quart and it is experimental/new).

As of late 2018 the Quart framework has split out an ASGI server called Hypercorn with Quart itself remaining as an ASGI framework. Hypercorn supports HTTP/2, meaning that Quart and any other ASGI framework can use Hypercorn to serve HTTP/2.

There are a number of frameworks exploring HTTP/2, including Tornado, Django (via channels and ASGI), and Sanic. Finally, a number of frameworks including aiohttp, Flask, Pyramid, and Falcon have no plans (or no plans I can find) to support HTTP/2.

Solutions

I’ll assume you are using a server that has Python 3.7 and openssl installed and you are familiar with pip and virtualenv.

Firstly, as no browser supports HTTP/2 without TLS[FAQ], we need to create a certificate. In production you should use a recognised authority such as Let’s Encrypt, but for this demonstration a self-signed certificate will be easier. To create a self-signed certificate follow this stack overflow answer i.e. enter the following command and accept the defaults,

$ openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes

As for the website itself, this article will borrow the Jumbotron Bootstrap 4 example since it can be condensed into a single html, css and js file. The full code and solutions are available on github.

The Quart solution

The simplest way to serve HTTP/2 is to use the Quart framework with a Hypercorn server, furthermore Quart is the only Python framework to support server-push. The solution itself is,

# quart_example.py
# pip install Quart
from quart import make_response, Quart, render_template, url_for

app = Quart(__name__)

@app.route('/')
async def index():
    result = await render_template('index.html')
    response = await make_response(result)
    response.push_promises.update([
        url_for('static', filename='css/bootstrap.min.css'),
        url_for('static', filename='js/bootstrap.min.js'),
        url_for('static', filename='js/jquery.min.js'),
    ])
    return response

if __name__ == '__main__':
    app.run(
        host='localhost', 
        port=5000, 
        certfile='cert.pem', 
        keyfile='key.pem',
    )

Readers familiar with Flask should recognise the above as a simple app (implicitly serving the static file due to the presence of a static folder) that has a single route, /, which returns the rendered index.html template. Unlike Flask, Quart has a push_promises attribute on its Response object, this is where the server-push paths are defined.

Viewing this in the browser gives the result we expect. Note the the browser security warning (due to the self-signed certificate) must be accepted.

Quart serving the website, note the use of server-push and h2 protocol.

As Quart recommends using Hypercorn to serve production data, the following commands can be used instead of the code within the if block in the example given above,

$ pip install hypercorn
$ hypercorn --keyfile key.pem --certfile cert.pem --bind 'localhost:5000' quart_example:app

The Twisted solution

The Twisted solution is almost as easy, thanks to this (slightly outdated) article,

# twisted_example.py
# pip install pyopenssl twisted[http2]

from twisted.web import server
from twisted.web.resource import Resource
from twisted.web.static import File
from twisted.internet import reactor
from twisted.internet import endpoints

if __name__ == "__main__":
    root = Resource()
    root.putChild(b'', File('./templates/index.html'))
    root.putChild(b'static', File('./static'))
    site = server.Site(root)
    server = endpoints.serverFromString(
        reactor,
        "ssl:port=5000:privateKey=key.pem:certKey=cert.pem",
    )
    server.listen(site)
    reactor.run()

The Twisted code puts a direct resource at the root path (an empty string corresponds to /) that returns index.html and another resource for the static folder including the files within.

When viewed in a browser this yields the expected result,

Twisted serving the website, note the use of the h2 protocol.

HTTP/1.1 solution

To compare to the equivalent with HTTP/1.1, by setting the alpn_protocols Hypercorn configuration to ["http/1.1"] instead of ["h2", "http/1.1"], gives,

Quart serving the website, note the use of the h11 protocol.

The advantages of HTTP/2 are hopefully clear, as even though these aren’t safe benchmarks the full transfer with HTTP/2 and server-push in 26ms whereas with HTTP/1.1 it was 38ms. This would improve dramatically if there were many more assets to fetch, as browsers are limited to the number of HTTP/1.1 requests can be made concurrently.

Conclusion

A little over two years since HTTP/2 was published it is now feasible to serve HTTP/2 responses including server-push in Python. As seen in the framework examples given, HTTP/2 requires a little additional effort compared with HTTP/1.1. Yet overall the Python support for HTTP/2 is limited with most frameworks offering no support and Twisted only as an extra. This will hopefully start to change.