Hybrid Processing: Deep Learning Comes to the Client Side

Nezare Chafni
Feb 2 · 2 min read

In a post-Covid world, there’s little doubt that future commerce will look to establish an online presence before considering a physical store. This latest shift (or acceleration) to the digital realm demands that governments and businesses around the world improve their online offerings, and even forces technophobes who were previously hesitant to perform tasks online to join the rest of us.

In consumer use-cases and apps, computer vision and deep learning are often performed on the server-side. Sometimes, this is done for practical reasons (the technology is computationally intensive) and other times it is done for security or operational reasons. But most often, it is done because we perceive it to be the simplest and most straightforward option.

The cloud is attractive because of traits like infinite scale and zero initial capex. But in many scenarios, a cloud-only approach can cause less than ideal experiences full of long feedback loops and user frustration (trying again after a few seconds of waiting for a sever response often leads to form abandonment).

Newer phones with specialized hardware and dedicated memory promise to solve the computation problem. But this imposes new requirements: forcing users to install a mobile app in order to be on-boarded, the need to convert your models to a supported framework (ex: CoreML or TensorFlow Lite), and the added complexity of having to do this differently for each platform.

Several JavaScript-based frameworks now make it possible to perform deep learning tasks directly in browsers. These frameworks provide a cross-platform solution to enable instant deep learning in web apps without the need for any installation or model downloads. The demo above shows a concept Spoof Detection capture app that can perform face detection, landmark localization, face size measurement, head pose estimation, intersection over union (IOU) measurement, face tracking, and point tracking… all completely in the browser at 20+ FPS. This allows us to ensure that a face meets a strict set of requirements and enhances the probability of success while improving the user experience.

In an era of instant apps and QR codes, users expect accessibility and speed with minimal friction. Adopting a hybrid processing approach (doing part of the processing on the client-side) promises to further improve the user experience and boost results. This capture app can be repurposed to enable and improve remote web-based enrollment (ex: scan QR code to enroll) as well as a host of other use cases.

Nezare Chafni

CTO @ Trueface.ai


For a safer and smarter world.