How to serve HTTP/2 using Python

HTTP/2, the latest version of the Hyper Text Transfer Protocol was published in 2015. It is at a high level compatible with HTTP/1 insofar as there are still requests to a method plus path and responses with a status code. Furthermore both requests and responses are accompanied with headers. At a lower level it is very different to HTTP/1, being binary rather than textual, introducing multiplexing, header compression and server-push among others.

The changes introduced by HTTP/2 primarily improve performance, especially page load times. Simply reasoned, a browser can now multiplex and pipeline multiple requests over a single connection rather than waiting for each connection in turn. In fact the browser may receive responses to requests it has yet to make via server-push.

This article aims to demonstrate the above mentioned HTTP/2 features by serving a simple website, consisting of html page and the associated css and js, via Python. Ideally the server will make use of HTTP/2 server-push to push the associated css and js to the browser, but at minimum it should pipeline the css and js requests.

HTTP/2 has pipelining, so you get a picture of pipes, although with multiplexing maybe there should only be the one pipe?

The simplest way to serve HTTP/2 is to use a reverse proxy (e.g. Nginx) to switch the protocol from HTTP/2 to HTTP/1 between the client and the Python server. This has some of the advantages to the client, e.g. multiplexing, however it cannot gain the full performance advantage and does not show how to serve HTTP/2 using Python.

The easiest way to serve a simple website using Python is to use a pre-existing web framework, however if you’d prefer to directly serve HTTP/2 you should consider the excellent hyper-h2 library and this example. It is also worth noting that almost all Python HTTP/2 implementations, including the frameworks given in this article, use hyper-h2.

Current framework state

As of late 2017 there are two Python frameworks that directly support HTTP/2, namely Twisted and Quart with only the latter supporting server-push (disclaimer I work on Quart and it is experimental/new).

There are a number of frameworks exploring HTTP/2, including Tornado, Django (via channels and ASGI), and Sanic. Finally, a number of frameworks including aiohttp, Flask, Pyramid, and Falcon have no plans (or no plans I can find) to support HTTP/2.

Solutions

I’ll assume you are using a server that has Python 3.6 and openssl installed and you are familiar with pip and virtualenv.

Firstly, as no browser supports HTTP/2 without TLS[FAQ], we need to create a certificate. In production you should use a recognised authority such as Let’s Encrypt, but for this demonstration a self-signed certificate will be easier. To create a self-signed certificate follow this stack overflow answer i.e. enter the following command and accept the defaults,

$ openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes

As for the website itself, this article will borrow the Jumbotron Bootstrap 4 example since it can be condensed into a single html, css and js file. The full code and solutions are available on github.

The Quart solution

The simplest way to serve HTTP/2 is to use the Quart framework, furthermore Quart is the only Python framework to support server-push. The solution itself is,

# quart_example.py
# pip install Quart
import ssl
from quart import make_response, Quart, render_template, url_for

app = Quart(__name__)

@app.route('/')
async def index():
result = await render_template('index.html')
response = await make_response(result)
response.push_promises.update([
url_for('static', filename='css/bootstrap.min.css'),
url_for('static', filename='js/bootstrap.min.js'),
url_for('static', filename='js/jquery.min.js'),
])
return response

if __name__ == '__main__':
ssl_context = ssl.create_default_context(
ssl.Purpose.CLIENT_AUTH,
)
ssl_context.options |= ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1
ssl_context.set_ciphers('ECDHE+AESGCM')
ssl_context.load_cert_chain(
certfile='cert.pem', keyfile='key.pem',
)
ssl_context.set_alpn_protocols(['h2', 'http/1.1'])
app.run(host='localhost', port=5000, ssl=ssl_context)

Readers familiar with Flask should recognise the above as a simple app (implicitly serving the static file due to the presence of a static folder) that has a single route, /, which returns the rendered index.html template. Unlike Flask, Quart has a push_promises attribute on its Response object, this is where the server-push paths are defined. Within the if block is the minimal acceptable SSL boilerplate for modern browsers and the key h2 ALPN protocol setting. The ALPN setting informs the browser that HTTP/2 can be used.

Viewing this in the browser gives the result we expect. Note the the browser security warning (due to the self-signed certificate) must be accepted.

Quart serving the website, note the use of server-push and h2 protocol.

As Quart recommends using Gunicorn to serve production data, the following commands can be used instead of the code within the if block in the example given above,

$ pip install gunicorn
$ gunicorn --worker-class quart.worker.GunicornWorker --keyfile key.pem --certfile cert.pem --ciphers 'ECDHE+AESGCM' --bind 'localhost:5000' quart_example:app

The Twisted solution

The Twisted solution is almost as easy, thanks to this (slightly outdated) article,

# twisted_example.py
# pip install pyopenssl twisted[http2]

from twisted.web import server
from twisted.web.resource import Resource
from twisted.web.static import File
from twisted.internet import reactor
from twisted.internet import endpoints

if __name__ == "__main__":
root = Resource()
root.putChild(b'', File('./templates/index.html'))
root.putChild(b'static', File('./static'))
site = server.Site(root)
server = endpoints.serverFromString(
reactor,
"ssl:port=5000:privateKey=key.pem:certKey=cert.pem",
)
server.listen(site)
reactor.run()

The Twisted code puts a direct resource at the root path (an empty string corresponds to /) that returns index.html and another resource for the static folder including the files within. Thereafter is a much reduced amount of boilerplate for the SSL settings as, unlike Quart, Twisted makes some sensible SSL assumptions.

When viewed in a browser this yields the expected result,

Twisted serving the website, note the use of the h2 protocol.

HTTP/1.1 solution

To compare to the equivalent with HTTP/1.1 the line in quart_example.py,

    ssl_context.set_alpn_protocols(['h2', 'http/1.1'])

can be changed to,

    ssl_context.set_alpn_protocols(['http/1.1'])

which yields,

Quart serving the website, note the use of the h11 protocol.

The advantages of HTTP/2 are hopefully clear, as even though these aren’t safe benchmarks the full transfer with HTTP/2 and server-push in 26ms whereas with HTTP/1.1 it was 38ms. This would improve dramatically if there were many more assets to fetch, as browsers are limited to the number of HTTP/1.1 requests can be made concurrently.

Conclusion

A little over two years since HTTP/2 was published it is now feasible to serve HTTP/2 responses including server-push in Python. As seen in the framework examples given, HTTP/2 requires a little additional effort compared with HTTP/1.1. Yet overall the Python support for HTTP/2 is limited with most frameworks offering no support and Twisted only as an extra. This will hopefully start to change.

Like what you read? Give Philip Jones a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.