WTF is Offline AI?

Offline AI refers to Artificial Intelligence programs that run on-device, as opposed to server-side APIs that run programs to perform AI tasks remotely. Why is this a thing? Well there are three big benefits to using Offline AI.


The first is operation speed. If a device has all the data it needs and possess the ability to perform intelligent tasks such as image recognition and natural language processing without needing to send/receive data processed on a remote server somewhere, then the speed of the operation is greatly improved due to the lack of reliance on network connectivity and/or server hardware performance.

An on-device AI program can run trained Machine Learning models and Neural Networks using nothing but the device hardware and software. Not having to rely on network connectivity greatly improves the speed of operation and has a positive impact on user experience. (Core ML for macOS and iOS is a framework by Apple that will allow such programs to run on a Mac or an iOS device. I’ll be talking more about Core ML in future posts).


The second benefit to using Offline AI comes hand in hand with the first. If Online is False then Network costs are Zero! To give you an example of how much of an impact this can have on an AI business..

The company I work for is an Artificial Intelligence and Augmented Reality company with a consumer facing mobile App, and it’s not uncommon for even medium-sized tech companies to spend millions of dollars per month on the server-side technology needed to perform the AI tasks that make an App’s feature-set possible.

The majority of this huge cost goes towards server hosting and data bandwidth fees that occur whenever the App sends image data from a user’s device camera and up to our online Neural Nets for processing. If you want really fast image-recognition performance for example, you’ll need to send up a lot of image data, multiple times per second. The promise of Offline AI eliminates this process altogether.


The third benefit is one of increasing importance to society as consumer technology and the social media industry matures, so is it that the responsibility to protect people’s data is more greatly required. User data privacy is an ethically important practice made possible by Offline AI.

Processing all data on-device means that it is sandboxed and better protected against data abuse and server hacking. Yes the device could still be hacked or stolen for that matter, but the risk of user data abuse is greatly reduced as the data is never sent to a remote network or stored server-side. User data can be processed, used for current tasks and then purged without leaving digital breadcrumbs when the data is no longer needed.

In time, as AI applications become more intwined with our daily lives, the need for this type of responsibility will increase and the onus is on us, program developers, software engineers and computer scientists to build such applications that behave respectively towards the personal security of the people that use them.

For more talks on Offline AI and specifically the use of Core ML in iOS mobile Apps, stick around ✌️