Introduction to Using Core ML

Preface

At present, all the leading experts in science and technology agree that artificial intelligence is the next frontier of technology. As a leader in the technology industry, Apple continues to lead the technology revolution, and introduced Core ML, a new machine learning framework. ML is the abbreviation of Machine Learning, which is a core tool in the field of artificial intelligence.

Core ML at a Glance

Core ML is the foundation for domain-specific frameworks and functionality. It enables developers to integrate a variety of machine learning models into their iPhone applications. In addition to supporting specialized deep learning for over 30 different applications, it also supports standard models such as tree ensembling, SVMs, and generalized linear models. Core ML is built on top of underlying technologies like Metal and Accelerate, so it seamlessly leverages the CPU and GPU to maximize performance. Machine learning models can be run directly on the device so that data is analyzed without detaching from the device.

In 2019, Apple published Core ML 3, which is the framework that powers cool features of the iPhone like FaceID, Animoji, and Augmented Reality. Core ML has come a long way since it was launched in 2017 and now supports a plethora of tools that can help us build great machine learning-based applications quickly.

Utilizing Core ML

Core ML provides a unified representation for all models, which encapsulate all the details of your machine learning model. Your app uses Core ML APIs and user data to make predictions, and to train or fine-tune models, all on the user’s device.

There are three ways to use Core ML models:

1. Embed into App

Drag the Core ML model(*.mlmodel) to your project.

xCode will generate model-related code automatically, including input, output and model definition. Click the arrow icon right of the model name, and it will show the code.

Then define the inputs and use the model to make predictions
Continue tuning and re-training the model until the target is reached.

2. Download from Network

Download and compile models within your app as an alternative to bundling with the app. Scenarios where this is a practical approach include:

• Reducing the app’s download size of your app on the App Store,
• Determining the right models for the user after installation, based on their location, specific interests, and so on.
• Providing model updates over the network.

Here are the steps in order to download and compile models within your app:

• Download the model definition file (ending in .mlmodel) onto the user’s device by using NSURLSession, CloudKit, or another networking toolkit.
• Compile the model definition by calling compileModelAtURL:error:.
• MLModel saves models it compiles to a temporary location. If your app can reuse the model at a later time, reduce your app’s resource consumption by saving the compiled model to a permanent location.
• Then construct input and use the model to make predictions, and to train or fine-tune the model.

3. Update by On-device Training

This basically means we train our model on some other machine and then utilize the trained model to make predictions in real-time on the device itself. This led to better user experience because we were not dependent on the internet to get predictions. Take Face ID for example. It needs to keep its model up-to-date when the user’s face changes over time (growing a beard, wearing different makeup, getting older, etc.). The basic idea is to initially have a generic model that gives an average performance for everyone, and then make a copy that is customized for each user.

With Core ML Tools, Core ML allows you to personalize models using the MLUpdateTask API. There are 3 types of models that are updatable:

1. neural networks
2. nearest neighbors
3. pipeline models

Generating Core ML Models

Use Create ML which is an IDE provided by Apple. Create ML lets you quickly build and train Core ML models right on your Mac with no code. The easy-to-use app interface and models available for training make the process easier than ever, so all you need to get started is your training data. You can even take control of the training process with features like snapshots and previewing to help you visualize model training and accuracy. It only supports apple defined models.

Image classifier
A model that has been trained to recognize images. When you give it an image, it responds with a label for that image.

Object detector
A model that has been trained to detect objects in images.

Sound classifier
A model that has been trained to classify sounds.

Activity classifier
A model that has been trained to recognize motions.

Text classifier
A model that has been trained to recognize text.

Word classifier
A model that has been trained to tag natural language text at the word level.

Tabular regresser
A model that has been trained for regression.

Tabular classifier
A model that has been trained for classification.

Recommender
A model that has been trained to provide recommendations.

Use Turi Create to create models. Turi Create is a library provided by Apple. It simplifies the development of custom machine learning models. You don’t have to be a machine learning expert to add recommendations, object detection, image classification, image similarity or activity classification to your app. With Turi Create, you can accomplish many common ML tasks:

Use third-party machine learning frameworks supported by CoreML. Then use Core ML tools to convert third-party models to Core ML models. Core ML tools contain all supporting tools for Core ML model conversion, editing and validation. This includes deep learning frameworks like TensorFlow, PyTorch, Keras, Caffe as well as classical machine learning frameworks like LIBSVM, scikit-learn, and XGBoost. With Core ML tools, you can do the following:

• Convert trained models from frameworks like TensorFlow and PyTorch to the Core ML format.
• Read, write, and optimize Core ML models.
• Verify conversion/creation (on macOS) by making predictions using Core ML.

Below is this list of models that Core ML 3 supports.

Conclusion

Core ML is Apple’s response to the new technological revolution that is artificial intelligence. Core ML delivers blazingly fast performance with easy integration of machine learning models, allowing you to build apps with intelligent new features using just a few lines of code.