Switching to a HTML5 Photo Manager: Bigger, Better, Faster, Stronger

By Horace Ko

Photos are an incredibly important component of listings on Airbnb, and have a demonstrable impact on the effectiveness of a listing; so much so, that we completely foot the cost for professional photographers to take photographs of hosts’ listings. It benefits both us and our users to make it easy as possible to upload and manage photographs of listings.

Our old photo manager was becoming a little long in the tooth. To upload a file, users would have to click on a standard HTML “file” input field, choose a single file in the Finder (or Explorer, on Windows) window, and then wait for the file to finish uploading and processing before choosing another file to upload. There was no indication of progress during this process; users would be stuck staring at a spinner until the response from the server was received. For large files, this spinner would stay up for quite some time, resulting in a frustrating experience.

We don’t like frustrating experiences.

Building a better experience

When the time came to do a once-over of our host tools, the photo manager received the bulk of our attention. We started by researching our users, and discovered that the majority of you — almost 70% — were using HTML5-capable web browsers built on the Gecko or WebKit rendering engine. As such, we decided to use a number of HTML5 technologies in implementing the photo manager, which allowed for such interactions as multiple-file drag-and-drop directly from your desktop, and asynchronous file uploads with progress state. We also relied on CSS3’s animation capabilities to smooth out some of the transitions (such as the filling of the progress bar), and to provide some fun feedback about the state of the photo manager (such as emulating the iOS “wiggle” when reordering photos).

Server-side changes

All of these changes in the UI/UX necessitated changes on the server side as well. Previously, all of the photo processing was done synchronously with the request, which increased the load on our nginx web frontends and resulted in elevated response times. We built a new photo processing backend using the delayed_job gem that relies on asynchronous job workers feeding off a queue of photo processing jobs. Our new backend allows us to scale the photo processing completely independently of our web frontends.

To further lighten the load on the web frontends, we decided to have the photo manager upload files directly to our Amazon S3 bucket. Performing cross-domain file uploads via AJAX requires implementation of the CORS specification in both the browser and the server. To provide some background, the CORS specification precedes any POST request to a server with an OPTIONS request, expecting the server to respond appropriately if CORS is allowed. If the expected response is not received, the browser blocks the followup POST request.

Targeting the photo manager towards HTML5-enabled browsers gives us CORS support in the browsers by default. Unfortunately, S3 does not respond appropriately. Inspired by a blog post on the subject, we worked around this problem by configuring our nginx frontends to respond directly to the browser’s initial OPTIONS request, and then proxy the followup POST request to S3. Here’s the configuration that we used:

Unlike an unproxied (direct) upload to S3, a proxied upload does consume some I/O bandwidth on our EC2 instances, but because we use nginx’s built-in mechanisms to proxy, it does save our Passenger/Rails web application layer from having to process the upload.

The End Result

The success of the new photo manager for listings meant that there was a big push to have it rolled out to other areas of the site, such as our user profiles and our new Stories feature, which we’ve been able to do quite easily. The new photo processing backend has been performing admirably, even with the increased load of multiple features utilizing it simultaneously.

Our focus on HTML5 browsers does beg the question: What do we do to support users on non-HTML5 browsers? In cases where a non-HTML5 browser is detected, we fall back to the old synchronous upload endpoint. Since we use CSS3 mainly to prettify transitions, the degraded state will also respond appropriately to user interactions.

We hope you enjoy our new HTML5 photo manager. Let us know what you think!

Check out all of our open source projects over at airbnb.io and follow us on Twitter: @AirbnbEng + @AirbnbData

Originally published at nerds.airbnb.com on December 11, 2012.