Mobile Photo Uploads with HTML5

Asynchronous file uploading on the web and especially the mobile web used to be a struggle. Three reasons come to mind:

• AJAX’s inability to send file data meant Flash and hidden frames became the go-to solutions
• Mobile browser support for Flash ranged from non-existent to extremely limited
• Many mobile operating systems — including older versions of iOS, Android, Windows Phone, and BlackBerry — lacked support in their browsers for the HTML input type=file attribute

Thankfully, the task of asynchronous file uploading on web and mobile web has been greatly simplified recently. In this post I share some considerations and details that went into the cross-platform, photo uploading app on our mobile web app.

Feature Detection vs Browser Detection

Many older mobile browsers lack support for HTML file input. Support on current devices is pretty good, but there are a significant number of old devices still in use that cannot handle HTML file input. In order to provide the best possible user experience, we can’t show users a feature their device cannot support — so we need to perform some sort of check to assess if the feature is supported.

One way of doing this is to check the browser’s user agent and cross reference that with a list of browsers or devices that are known (either by manual testing or documentation; caniuse is a nice place to start) to support the feature. This is known as browser detection, and is problematic for many reasons. For one, it is difficult to verify all possible device/browser configurations that support the feature of interest. Additionally, it isn’t future-proof: the code you write today may not work tomorrow.

For these reasons, it is much better to detect if the feature of interest is available to use in the user’s browser, a practice known as feature detection. This is the goal behind the popular open source JavaScript library Modernizr. The ideal solution would use only feature detection without any browser detection; however, this is not always possible since many browsers will report false positives on certain features.

The compromise is to use a mix of the two methods when using only feature detection is not possible.

Below is a snippet of code adapted from Modernizr that detects support for HTML file input using a mix of feature and browser detection:

Selecting a File

Once we know that file input is supported, we can show the HTML input field to the user. When tapped on a mobile device, this input field will prompt the user to either take a new photo with the camera, or to select an existing photo on their device. It’s also possible to allow the user to select multiple photos at once on some devices, by adding the multiple attribute to the input element; however, be aware that on iOS this will remove the option to use the camera as a source.

Note that here we use client-side validation to ensure that the selected file is an image, but it is important to validate on the server side as well. In general, validation on the client side provides better feedback to the user, whereas server-side validation is important for security purposes.

[caption id=”attachment_2452" align=”aligncenter” width=”450"]

A look at the file input dialogue on iOS and Android[/caption]

Reading and Processing the File

The improvement of camera technology in mobile devices brings with it an increase in the size of photos. Unfortunately, most people don’t have the bandwidth (or the patience) to upload uncompressed 8+ megapixel photos on a regular basis!

So what can we do? One solution is to compress and/or downscale photos on the client side before we send the photo over the network. In the past this would not have been a trivial task, but the introduction of HTML5, the FileReader interface, and the Canvas API makes this fairly simple. Let’s break this down into two parts:

1. Using the FileReader interface to read the file
2. Using the Canvas API to downscale and compress the image

1. Reading the File

In order to read the file, we first need to listen for changes to the HTML input element described earlier. We can do this with a little jQuery (although the native JavaScript implementation is not much different):

In this step it can also be worthwhile to validate that the file is under a given size, as loading a very large image into memory with the Canvas API is expensive, slow, and could cause the browser to crash. When the user selects an image file, the readFile function will be called with the file object. To read the file, we create a new FileReader object and define the success and failure conditions of reading. We must read the file in one of several different formats including array buffer, binary string, data URL, and text. You can refer to the FileReader documentation for details on each of these, but for our purposes reading the file as a data URL is fine as it works quite seamlessly with the Canvas API.

2. Processing the File

Once reading the image has completed, we can process it with Canvas. First we create a new image and set its source to the data URL obtained from reading the file. When the image has loaded, we calculate the desired dimensions of the new image. Note that if the size of the new image is smaller than the original, there is no need to process the image (if scaling is our only objective).

Next, we create a canvas element which will hold our output image. The canvas getContext method returns an object that provides methods and properties for manipulating the canvas. We use the drawImage method to place the source image on the canvas, resulting in a downscaled version of the original image.

The canvas element provides two methods to retrieve image data: toDataURL and toBlob. Both toDataURL and toBlob take as arguments the output image type and the image quality. If the image type is specified as image/jpeg or image/webp, the quality argument (a number between 0 and 1 with a default of 0.92) can be used to further compress the image.

This is a fairly simple use case of the Canvas API, though the possibilities extend far beyond scaling. Some examples include client side image editing, cropping, and thumbnail generation. An issue particularly relevant to the mobile web is the problem of lost metadata: most browsers will ignore the EXIF metadata of photos, including the orientation tag, resulting in photos being oriented incorrectly. This is not a problem if we upload the original photo, but since we are generating a new image with Canvas, the EXIF data of the original photo is lost. One solution is to read the EXIF data on the client side and use Canvas to correctly orient the image.

Uploading the File

In recent years, the technology behind AJAX has evolved to be able to handle file uploads, so we take advantage of this to upload the file asynchronously. There are many benefits to asynchronous file uploading, including the ability to monitor the progress of the upload, and allowing the user to perform additional actions while they wait for the upload to finish. Additionally, if the upload fails for whatever reason, we can provide the user with a hassle-free way of retrying the upload:

[caption id=”attachment_2451" align=”aligncenter” width=”300"]

Tap to retry a failed upload on Hootsuite’s mobile web app[/caption]

To keep things simple, we will use jQuery’s ajax method to perform the request, but if you are familiar with the XMLHttpRequest (XHR) object this task should be no more difficult.

We use a FormData object to construct the data payload that we want to send to the server. Using the FormData append method, we can add key/value pairs for files, blobs, and strings to the data payload. When using jQuery and FormData, it is important to set the processData and contentType properties to false. Setting processData to false prevents jQuery from automatically converting the data into a string, while setting contentType to false will prevent jQuery from incorrectly setting the content type to its default of application/x-www-form-urlencoded.

Once the file data reaches the server we can handle it with the backend technology of our choice. In PHP, the file data can be accessed in POST if the file data was sent as a data URL, or FILES if the file data was sent as a file or blob.

Conclusion

While asynchronous file uploading on the mobile web has long been a headache, the introduction of XMLHttpRequest Level 2 and various HTML5 APIs make this task much much easier. This post only covers the logic behind file processing and uploading, but with a bit of styling and polishing it’s quite easy to provide an awesome user experience. Check out the final product on Hootsuite’s mobile website.

[caption id=”attachment_2454" align=”aligncenter” width=”211"]

Photo sharing via Hootsuite’s mobile web app[/caption]

Thanks

Thanks to Adam Arsenault, Lars Vedo, Jeff Stautz, Kimli Welsh, and Noel Pullen for their advice and valuable feedback.

Further Reading

• Viljami Salminen’s Blog — File Upload Support on Mobile
• Mozilla Developer Network — FileReader Documentation
• Mozilla Developer Network — Canvas Documentation
• Flickr — Parsing Exif client-side using JavaScript

About the Author

Jacob Lee is a co-op student on Hootsuite’s mobile web team, studying computer science at UBC. When he’s not creating web and mobile apps, he enjoys tinkering with digital media, playing hockey and tennis, and jamming on the guitar. Follow him on Twitter @jacobtwlee.