Using http/2 for App Engine Local Development
I’ve been using App Engine for many years to develop web apps at Google. Most of the open source projects I’ve worked on use it for its simplicity and scalability. A couple of examples are the Google I/O web app, the Chrome team’s Chromestatus.com, Polymer’s site, developers.chrome.com, and WebFundamentals, just to name a few. Heck, I’m even using it to build my wedding website!
http/2 brings development and production closer together
I could say a ton of nice things about App Engine, but one of its huge drawbacks is that the local development environment is far from Google’s production environment (where your app actually runs). One of the most important differences is that App Engine’s development server uses http/1.1 and Google’s infrastructure uses http/2 (h2).
Besides being 16 years in the making, h2 offers many performance improvements over it’s 1.1 predecessor: multiplexing, header compression, server push). In a nutshell, this is difference between the two:
One of things I’m most excited about is that h2 makes development closer to production. By that I mean the shape of a web app doesn’t change when we hit the deploy button.
If you’re like me, you probably develop source code in small, individual files. That’s good for organization, maintainability, and our general sanity :) But for production, we create an entirely different story. We roll sprite sheets, domain shard, concatenate CSS, and bundle massive amounts of JS together to squeeze out every last drop of performance. With h2, all of those techniques become a thing of the past; and in fact, anti-patterns.
The h2 protocol can make it more efficient to serve many small files rather than a few large ones.
Small, individual files can lead to improved performance through better HTTP caching. For example, a one-line change won’t invalidate 300KB of bundled JavaScript. Instead, a single file gets evicted from the browser’s cache and the rest of your code is left alone.
So…http/2 is pretty great. All major browsers support it and large cloud/CDN providers have finally started to bake it in. The place that’s still lacking is our development setups (remember my peeve about keeping dev ~= prod).
Since I use App Engine all the time, I wanted a way to close the gap between its prod and dev environment and utilize http/2 on App Engine’s dev server. Turns out, that’s not too hard to do.
Enabling h2 with the App Engine dev server
It’s hard to performance tune an app when the HTTP protocol it uses locally is different than that of production. We want both environments to be as close as possible to each other.
To get dev_appserver.py serving resources over h2, I setup a reverse proxy using a server that supports h2 out of the box. I recommend nginx because it’s fast, easy to setup, and easy to configure. The second thing we’ll need to do is setup localhost to serve off https. That sounds scary, but it’s fairly straightfoward
SSL is not a requirement of the h2 protocol but all browsers have mandated it for http/2 to work and many new (service worker) and old (getUserMedia, geo location) web platorm APIs are requiring it.
Setting up Nginx as reverse proxy to App Engine
First, I installed nginx using Homebrew:
brew install --with-http2 nginxNginx is supposed to support h2 out of the box since v1.9.5, but I had to install it using —with-http2 to get the goodies.
Homebrew installs nginx to ~/homebrew/etc/nginx and will server static assets from ~/homebrew/var/www/. The install also creates a ~/homebrew/etc/nginx/servers directory where you can stick custom server configurations.
To add a server, create ~/homebrew/etc/nginx/servers/appengine.conf with:
server {
listen 3000;
server_name localhost; # If nginx cant find the requested file,
# forward it to the GAE server.
location / {
try_files $uri @gae_server;
} location @gae_server {
proxy_pass http://localhost:8080;
}
}
What this does is forward any requests that nginx can’t find (right now that’s all of them) to your GAE app running on 8080.
Next, fire up your GAE app on 8080:
cd your_gae_app;
dev_appserver.py . --port 8080and start/stop nginx:
nginx
nginx -s stopAt this point, you should be able to open http://localhost:3000/ and see your GAE app! Requests are still over http/1.1 because we haven’t setup SSL yet.
Enabling SSL for localhost (ngnix)
First, generate a self-signing certificate in ~/homebrew/etc/nginx/:
sudo openssl req -x509 -sha256 -newkey rsa:2048 \
-keyout cert.key -out cert.pem \
-days 1024 -nodes -subj '/CN=localhost'This will create a private key (cert.key) and a certificate (cert.pem) for the domain localhost.
Next, modify appengine.conf like so:
server {
listen 443 ssl http2;
server_name localhost; ssl on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_certificate cert.pem; # or /path/to/cert.pem
ssl_certificate_key cert.key; # or /path/to/cert.key location / {
try_files $uri @gae_server;
} location @gae_server {
proxy_pass http://localhost:8080;
}
}
The first couple of lines enable the ssl and http2 modules on localhost:443. The next few instruct the server to read your private key and certificate (the ones you just generated). The rest of the file remains the same as before.
The OS will throw permission errors for opening ports under 1024, so you’ll need to run nginx using sudo this time. The following command worked for me but you might be able to get away with just sudo nginx:
Start nginx using sudo:
sudo ~/homebrew/bin/nginxOpen https://localhost/ (note the “https”) and you’ll get a big ol’ security warning from the browser:
Don’t worry! We know that we’re legit. Click “ADVANCED”, and then “Proceed to localhost (unsafe)”.
If you really want the green lock, check out the instructions here to add the self signed certificate as a trusted certificate in MacOS System Keychain.
Hitting refresh again should give you responses over h2:
Take this in. Your local GAE app is running over SSL and using http/2 to serve requests!
What about h2 server push?
Tip: see my drop-in http2push-gae library for doing h2 push on Google App Engine.
At the time of writing, Nginx doesn’t support h2 server push, but that doesn’t mean we can’t test with it locally!
h2o is another modern h2 server that’s even easier to configure, comes with an up-to-date h2 implementation, and supports server push out of the box.
First, install h2o using Homebrew:
brew install h2oBy default, h2o installs to ~/homebrew/bin/h2o and will serve static files from ~/homebrew/var/h2o/. You can change where files are served by editing ~/homebrew/etc/h2o/h2o.conf.
Start a web server and verify that you see the default index.html page on http://localhost:8080/:
h2o -c ~/homebrew/etc/h2o/h2o.confNext, create a new server config, ~/homebrew/etc/h2o/appengine.conf:
hosts:
"localhost":
listen:
port: 3000
paths:
"/":
proxy.reverse.url: http://localhost:8080/In the example, I’ve done the same thing as the ngnix setup. We’ve setup a server on port 3000 that will forward all requests to App Engine running on port 8080.
Enabling SSL for localhost (h2o)
First, copy over your cert and key from the ngnix steps above (or generate new ones):
cp ~/homebrew/etc/nginx/cert.key ~/homebrew/etc/h2o/
cp ~/homebrew/etc/nginx/cert.pem ~/homebrew/etc/h2o/Modify ~/homebrew/etc/h2o/appengine.conf to include an entry for localhost:443:
hosts:
"localhost:443":
listen:
port: 443
ssl:
certificate-file: cert.pem
key-file: cert.key
paths:
"/":
proxy.reverse.url: http://localhost:8080/Start the server using sudo (again, because we’re opening a special port, 443):
sudo ~/homebrew/bin/h2o -c ~/homebrew/etc/h2o/appengine.confBe sure you’ve started the GAE dev server (dev_appserver.py . — port 8080), and open https://localhost to see your running GAE app. Any resources that contain a Link rel=preload header will be server pushed:
If you want to determine if a resource is being pushed, look for the x-http2-push: pushed header in the response. h2o will set that header on pushed resources. Alternatively, you can drill into Chrome’s chrome://net-internals page to verify pushed resources.
Maximize perf: speeding up static resources
If you want even more speed, you can have nginx or h2o serve your static files directly instead proxying them to the dev server. Both servers are much faster than dev_appserver.py and will better mimic production App Engine.
Configuring Ngnix to server static resources
Add root /path/to/gae_app/src; to your server config:
server {
listen 443 ssl http2;
server_name localhost;
root /path/to/gae_app/src; # add this ssl on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_certificate cert.pem;
ssl_certificate_key cert.key; location / {
try_files $uri @gae_server;
} location @gae_server {
proxy_pass http://localhost:8080;
}
}
If nginx can find the file within your root, it will serve it directly rather than (needlessly) forwarding it to App Engine. All other requests will be proxied to App Engine as usual.
Configuring h2o to server static resources
Likewise, h2o can be instructed to serve your static files using file.dir. Just specify a URL request -> /path/to/src mapping:
hosts:
"localhost:443":
listen:
port: 443
ssl:
certificate-file: cert.pem
key-file: cert.key
paths:
"/":
proxy.reverse.url: http://localhost:8080/
"/static":
file.dir: /path/to/gae_app/static # add thisNow, all files under https://localhost/static/* will be served by h2o instead of GAE.
Tip: Check your dev server logs to confirm ngnix/h2o are handling the static files. If requests don’t show when you refresh the page, you’re good to go. If requests show up, check that you’re using the correct path for root or file.dir.
And with that, Voila! We’ve got the App Engine development server running fully over http/2.
Credits
These were invaluable resources when researching this post:
Originally published at ericbidelman.com.
