Getting Started with Core ML — ML on iOS

▶ OVERVIEW

▼ GETTING STARTED WITH CORE ML ON iOS

Now that we have an overview of Machine Learning on iOS, let's get our hands dirty! We’ll start with the basics: Getting an iPhone to recognize objects using the camera and Core ML.

Step 1. Feeding camera input to a Core ML model

Here we cover how to access the camera and use that as input for a CoreML model. A CoreML model is a pre-trained Machine Learning model that can be read by iOS.

Duration

• 15–45 minutes 🕧

Requirements

• Xcode 9 or above
• iOS 11 device compatible. Ideally one that can run ARKit (iPhone 6s, 6s+, 7, 7+, 8, 8+, X, SE. iPad 2017, iPad Pro (any))

Tutorial List

Below you’ll find a curated list of tutorials on how to load a CoreML model onto your iPhone.

( Author note: As there were plenty of tutorials covering this step, I’ve opted not to write my own for now. I’m too excited getting the next section ready where we’ll be training your own CoreML model! )

Skim through these tutorials, pick one, and try it out.

• “VisionKit and CoreML” — Chris Greening
• “Building an object recognizer iOS app using CoreML, Vision and Swift” — George Jose
• “How to build an image recognition iOS app with Apple’s CoreML and Vision APIs” — Mark Mansur
• “CoreML: A Quick Walkthrough” — Nick Bourdako
• “Core ML Tutorial: Create a Simple Machine Learning App — Image Classification” — Brian Advent’s Youtube
• “Introduction to Core ML: Building a Simple Image Recognition App” — Sai Kambampati (AppCoda)
• “Core Machine Learning for iOS Developers”– Ayoub Khayati

If you’re overwhelmed, I would recommend Chris Greening’s tutorial on CoreML.

Chris Greening’s tutorial on Core ML

Image access on iOS

There are 3 popular ways of feeding an image into CoreML:

• ARKit — Simple to set up and usable with Augmented Reality. Currently suitable for small models (under 200 MBs), as larger models can cause ‘hiccups’ to the camera feed. These hiccups may disappear in the future as Apple continues to tweak ARKit.
• AVFoundation — The tried and true method of accessing the camera in realtime. This is suitable for models of any complexity. Requires many lines to set up.
• UIImagePicker — Allows a user to choose a photo from their photo library. Not suitable for real-time applications.

Don’t worry if your tutorial didn’t have ARKit, we’ll be covering it in our next article.

Optional Reading

If you’re new to Machine Learning, I’d recommend checking out Apple’s ML Introduction: WWDC 2017 — Introducing Core ML (31 minutes).

Apple WWDC 2017 — Introducing Core ML — Session 703

If you’re an intermediate coder (i.e. if you’ve heard of Python, AVFoundation, Tensor Flow), I recommend checking out this video by Alexis Gallagher: “Everything a Swift dev needs to know about Machine Learning” (21 minutes).

Next

Next: Entire pipeline, from Data to iOS App

The models in the above list of tutorials are generic models that tend to recognize many objects at the expense of accuracy. If we want an application that can accurately recognize our own objects, we’ll need to feed in our own data.

So next, we’re going to create our own CoreML model to classify our own objects. Excellent for recognizing pets 🐁🐠🐕🐘, plants 🌸🌻🌺🌵 or to See Food 🌭.

▶ CREATE YOUR OWN OBJECT RECOGNIZER

▶ CONVERSION… (WIP)

▶ TRAINING… (WIP)