Building 60 FPS QR Scanner for the Mobile Web

Jacky Efendi

Follow

Published in

Tokopedia Engineering

6 min readJan 8, 2020

--

Nowadays, web-apps are expected to be more and more powerful. Since the conception of AJAX, the web is no longer a medium to simply view HTML documents. People are building an actual full-fledged app to run on web browsers! These are made possible by the ever-growing list of new web APIs, the massive amount of modules written by the community, and the collaboration between browser vendors.

The Goal 🥅

Our goal was to build a QR scanner for our mobile web. Many e-commerce platforms have QR scanner. Tokopedia has had one in their native mobile apps for a long time, but not in the mobile web. The reason was not that it was not possible to implement. There are many libraries out there written by the community that handle this, but because image processing is an inherently resource-intensive task, it is hard to maintain a good user experience while doing it. So, we used this opportunity as an excuse to start looking into WebAssembly.

WebAssembly (Wasm) 🛠

[WebAssembly] provides a way to run code written in multiple languages on the web at near native speed, with client apps running on the web that previously couldn’t have done so. — MDN

WebAssembly allows us to take a code written in another language, compile it to .wasm and then use it alongside our JavaScript in the browser! There are many languages that can compile to .wasm, not just C, C++ and Rust. There is even something called AssemblyScript, which allows you to write TypeScript and compiles to WebAssembly! You can check out the long list of languages here.

How is WebAssembly faster than JavaScript though? I will not bore you with the details, but in short, it is because:

JavaScript is inherently dynamic, making it hard for compilers to optimise them
Different browsers can run different JavaScript engines, which can have their own different ways to optimise the JavaScript they run
JavaScript can be as fast as WebAssembly (for now), but it can very easily be de-optimised to the slow path

If you would like to understand more, I highly recommend watching the following talks: Surma’s WebAssembly for Web Developers and Franziska Hinkelmann’s JavaScript engines — how do they even?.

The Experiment 🤔

So, we started doing some benchmarking to compare the difference between using JavaScript and WebAssembly to do QR decoding. We found a JavaScript library named jsQR and C library named quirc. We then wrote some code in C that uses quirc to do QR decoding, and compile it to WebAssembly using emscripten. Emscripten will output a .wasm file and a .js file containing glue codes for interacting with the .wasm file.

Benchmark for jsQR vs. quirc.wasm (average main thread time)

The chart shows the time it took to do QR-decoding on an image on a MacBook Pro 2018, with 6x CPU slowdown. jsQR took an average of around 47ms, while quirc.wasm took an average of around 29ms.

Another interesting thing is, in some cases, our scanner that was using jsQR jumped to a whopping 1023ms! It then averaged at around 800ms.

Benchmark for jsQR vs. quirc.wasm (min and max main thread time)

This finding is in line with what we learned from Surma’s talk, that JavaScript can be de-optimised and unpredictable compared to WebAssembly.

Finally, these numbers also translate to the FPS we get.

We get around 34 FPS on quirc.wasm and around 17 FPS on jsQR. The numbers of course will vary depending on the device and browser, but I think we can safely conclude that doing QR decoding with WebAssembly will yield better performance.

Offloading it to Web Worker 👷

Web Workers are a simple means for web content to run scripts in background threads. The worker thread can perform tasks without interfering with the user interface. — MDN

Using web workers, we can do task in a separate thread so the main thread does not get fully occupied with all tasks. This concept is similar with what native apps have been doing, having a main UI thread to handle UI, and handling other tasks in separate thread. Here is another great talk about why you should try finding things to offload to workers: The main thread is overworked and underpaid (Chrome Dev Summit 2019).

To achieve 60 FPS, the main thread has to ship frames every 16.7 ms. This is very hard to do especially if the main thread has to do QR decoding as well. Using WebAssembly helped making the process fast enough, but it still runs on the main thread.

We ran emscripten again with different options so the output can be run in workers. To make it easier to use in our webpack-built-app, we used comlink-loader. It allowed us to have an easy-to-use API when working with workers.

NOTE:
If you are looking to use workers in your web-app, the author (Jason Miller) suggested to use worker-plugin + comlink instead. The reason we are still using comlink-loader is only for the inline capability.

The Results 📈

Benchmark including running wasm inside worker (average main thread time)

Doing the QR decoding with WebAssembly +Web Worker allows our main thread to be less occupied. For every decoding, the main thread only took around 6ms. Note that the main thread does not actually do any decoding. It just passes the image data to the worker and receives the result. Overall, the QR decoding is slightly slower because of the overhead of passing around data between the main thread and the worker thread. But, the benefit of having our main thread less busy is worth the trade. It allowed us to achieve 60 FPS for our QR scanner! 🎉

Benchmark including running wasm inside worker (average FPS)

Summary 📝

Having WebAssembly and Web Worker in our arsenal can really help us build powerful web-apps. We can do a lot of interesting stuffs in browsers, while still keeping the user experience great. The great tools available today such as emscripten, comlink, worker-plugin, comlink-loader, and webpack really help making the experience of doing all these pleasant! If you have not played around with WebAssembly and/or Web Worker before, there hasn’t been a better time to start doing so!