A barcode scanner
eBay native apps both iOS and Android have a barcode scanner feature in the selling flow. The feature leverages the device camera to scan a product UPC barcode and automatically fill out the listing, thus removing the manual overhead. This was a native app-only feature. It requires some intense image processing on the device to detect the barcode number from the camera stream. The retrieved code is then be sent to a backend service which, in turn, fills out the listing. This means that the on-device image processing logic has to very performant. For native apps, we compiled an in-house built C++ scanner library into native code for both iOS and Android. It was extremely well performant in generating the product barcode from the camera stream. We are slowly transitioning to iOS and Android native APIs, but the C++ library is still solid.
The barcode scanner is an intuitive feature for our sellers, as it made the listing flow more seamless. Unfortunately, this feature was not enabled for our mobile web users. We already have a well-optimized selling flow for the mobile web, except that the barcode scanner was not available, and sellers have to manually enter the product UPC, thus adding more friction.
A barcode scanner for the web
One option is to wait for the Shape Detection API. This proposed web API brings many native image detection features to the web, one of which is barcode detection. It’s still in very early stages and still has a long way to achieve cross-browser compatibility. Even then, it is not guaranteed to work on every platform. So we have to think about other options.
This is where WebAssembly comes into play. If a barcode scanner is implemented in WebAssembly, we can make strong guarantees that it would be consistently performant. The strong typing and structure of WebAssembly bytecode enable the compilers to always stay on the hot path. On top of it, we had an existing C++ library that was doing the job for native apps. C++ libraries are ideal candidates to be compiled to WebAssembly. We thought we had a clear path. Well, not exactly.
Our engineering design to implement a WebAssembly-based barcode scanner was pretty straightforward.
- The main thread will send a snapshot from the camera stream to the worker, and the worker will call the corresponding wasm API through the glue code. The response of the API is passed to the main thread. The response can either be the UPC string (which is passed to backend) or an empty string if no barcode is detected.
- For the empty scenario, the above step is repeated until a barcode is detected. This loop is timed by a configurable threshold in seconds. Once the threshold is reached, we display a warning message “This is not a valid product code. Please try another barcode or search by text.” This either means the user is not focusing on a valid barcode or the barcode scanner is not performant enough. We track these timeout instances, as it is a good indicator of how well the barcode scanner is performing.
The first step for any WebAssembly project is to have a well-defined compilation pipeline. Emscripten has become the de facto toolchain for compiling WebAssembly, but the key is to have a consistent environment that produces a deterministic output. Our frontend is based on Node.js, which means we need a solution that works with the npm workflow. Fortunately, it was around the same time that the article “ Emscripten and npm” was published by Surma Das. The Docker-based approach for compiling WebAssembly makes perfect sense, as it removes a ton of overhead. As recommended in the article, we went with the Docker Emscripten image by trzeci. We had to make a couple of tweaks to the custom C++ library to make it compatible to compile to WebAssembly. That was mostly a trial and error exercise. Ultimately we were able to compile and were also able to set up a neat WebAssembly workflow within our existing build pipeline.
It was fast, but…
The way we calculate the performance of the scanner is by analyzing the number of frames the wasm API can process in a second. The wasm API takes in a frame, in this case, an image snapshot pixel data from the live camera stream, performs the calculations and returns a response. This is done on a continuous basis until a barcode is detected. We measure it in terms of the well-known Frames Per Second (FPS) metric.
Initially, we were clueless about why the custom C++ library, which worked perfectly well for native apps, did not produce the same result for the web. After a lot of testing and debugging, we found that the angle in which we focus the object, along with the background shadow, determines the time for successful detection. Then how did it work in native apps? Well, in native apps we use inbuilt APIs to either autofocus or provide user tap focus to the center of the object that is being scanned. This enables native apps to send high-quality image pixel data (i.e. information only about the barcode) to the scanner library at all times. This avoids the blurry image situation. Hence the consistently fast response times.
Now that we had an idea about what is going on, we thought maybe a different native library might perform better under varied focus conditions. The open source barcode reader ZBar is pretty popular and stable. More importantly, it works well with blurry and grainy images. Why not try that? Since we have a WebAssembly workflow already set up, compiling and deploying ZBar as WebAssembly was seamless. We then began evaluating the ZBar implementation. The performance was decent, around 15 FPS (not as good as our custom C++ lib). However, the success rate was close to 80% for the same timeout threshold. Definitely an improvement over our custom C++ library, but still not 100% reliable.
We were still not satisfied with the outcome, but we noticed something unexpected. The scenarios in which ZBar timed out, the custom C++ library was able to get the job done very quickly. This was a sweet surprise. Apparently, based on the quality of the image snapshot, the two libraries performed differently. This gave us an idea.
Multithreading and racing to the rescue
You probably guessed it. Why not create two web worker threads — one for ZBar and one for the custom C++ library — and race them against each other. The winning response (i.e. the first one to send a valid barcode) is sent to the main thread, and all workers are terminated. We set this up and started internal dogfooding to simulate as many scenarios as possible. This setup yielded us a 95% success rate when scanning a valid barcode. Much better than our previous success rates, but it still falls short of 100%.
The following illustration shows a high-level flowchart:
A note about asset loading
After thorough testing and internal dogfooding, the feature was launched as an A/B test. The “Test” bucket in the experimentation showed the barcode scanner icon (screenshot below) and the “Control” did not.
The metric that was used to evaluate the success of the A/B test was something called “Draft Completion Rate.” It is the rate at which a listing goes from a draft stage to successfully completed and submitted. The Draft Completion Rate is a good indicator to support the notion that reducing friction and proving a seamless selling flow through a barcode scanner should enable more listings to be completed. We ran the tests for a couple of weeks, and when the results came back it was indeed very satisfying. It perfectly aligned with our original hypothesis. The Draft Completion Rate improved by 30% for the listing flow with a barcode scanner enabled.
The whole WebAssembly journey was a great learning experience for us. Engineers get pretty excited about new technologies and immediately want to try them out. If the same technology makes a positive impact on a customer-centric metric, it is a double delight. This alludes to an earlier point in this post. Technology evolves at a very rapid pace. Every day we hear new things getting launched. But only a few make a difference to customers, and WebAssembly is one of them. This was our biggest learning from this exercise — “Saying “No” to 99 things and “Yes” to the one thing that really matters to our customers.”
As next steps, we are looking into expanding the barcode scanner to the buying side of the mobile web, which would allow buyers to scan items to search for and purchase. We will also look into augmenting this feature with the Shape Detection API and other in-browser camera capabilities. Meanwhile, we are happy that we found the right use case for WebAssembly at eBay and in bringing the technology to ecommerce.
Originally published at https://www.ebayinc.com on May 22, 2019.