Machine Learning on Smartphones in 2017
Machine Learning is transforming internet applications. Our mail boxes, social networking sites, shopping sites currently leverage the power of machine learning to enhance our experience and the shift has been very progressive. It allows the applications to deliver a very intuitive and personalized experience which elevates the user experience to a completely new level. Soon machine learning will transform the mobile applications we use and this article provides a glimpse of the potential it holds.
Traditional way to use a computer to perform image classification task would involve a developer studying the image properties and explicitly identifying features in the image for labeling it. Naturally traditional way was extremely hard in as the image features were very specific and the algorithms used fell short of providing high accuracies as the number of items to identify increased. Machine learning instead uses a more natural way. In machine learning approach we provide the algorithms with data and its labels and the algorithms derive the features which can be used for classification. Therein lies the power of machine learning. These advanced algorithms are now dependent on data instead of specific features and with current proliferation of internet gathering data is more convenient.
Machine learning finds its use in almost every application due to versatile nature of algorithms. From targeted advertisements based on user internet history to cybersecurity by scrutinizing server logs it’s an ever growing platform. These algorithms need large amount of data and require huge computational resources during training stages. These two factors have thus restricted training and use of machine learning models on server hardware. The most common architecture is to use server equipment for heavy lifting tasks such as creating models and create API interface for client applications to interact with the model.
With recent progress in field of machine learning platforms some tasks can now be carried out on the client devices which has multiple advantages. Specifically these platforms now allow the trained models to be ported onto the mobile device and provide an interface to mobile applications to query the model. As the applications can now interact with the models right on device there is no dependency on network and there is a huge gain in app performance. Also some machine learning platforms allow models to be trained using only the device hardware which has the potential to eliminate the need for server-client model.
Moving from the platform towards specific applications there are multiple use cases wherein machine learning can improve mobile applications. As an early adoption of these algorithms advanced features were added to smartphone keyboards such as predictive text and auto-correction of spellings. Soon these features were enhanced with machine learning techniques to adapt to the usage patterns of individual users and provide more intuitive results. Other applications which were early adopters include smart device assistants such as iPhone’s Siri and Google Now. Soon companies started utilizing machine learning algorithms to organize photos into albums based on various learned data points such as date and time, location, people and objects in the pictures. The photos can then be easily converted into photo stories or make pictures searchable by what’s in them. These early adoptions uploaded the user data to their servers and performed the heavy lifting tasks of training models on the server hardware and utilized mobile devices as clients.
Broadly machine learning can be used in various domains on mobile devices such as Natural Language processing (NLP), Image and video processing, Voice detection and manipulation, authentication, battery optimization and much more.
One of the most essential component of smartphone application is authentication. With power of machine learning algorithms the data collected by motion sensors, location sensors, touch inputs as well as application usage can be utilized to create a user profile on the device. This profile can then be used to authenticate users on the device which will result into frictionless access for authorized users as well as robust resistance to impersonators. Such an implementation will by design keep private user data on the device and make the system resilient to server side data breaches and security attacks.
Moving towards current tools and platforms for implementing machine learning in mobile applications include TensorFlow developed by Google which works on both iOS and Android and Core ML developed by Apple which works on iOS devices only.
Here are some interesting links to get started with implementing machine learning on mobile devices:
1. Tensorflow
Tutorial 1 :
Tutorial 2:
2. Machine learning on iOS
I believe this is the beginning of a new era in machine learning development. Progress in machine learning on smartphones will empower developers to create much better applications and enrich our experience as we step into the future.
A few references to read further:
1. Machine learning is going mobile
2. Deep Learning on smartphones
3. Dive deeper into what’s supported in CoreML
4. A Guide to CoreML on iOS (Building spam detector)
5. Core ML and Vision: Machine Learning in iOS 11 Tutorial
6. Supported core ML models which can be ported to smartphone
