Getting Started with Core ML — ML on iOS
▶ OVERVIEW
▼ GETTING STARTED WITH CORE ML ON iOS
Now that we have an overview of Machine Learning on iOS, let's get our hands dirty! We’ll start with the basics: Getting an iPhone to recognize objects using the camera and Core ML.
Here we cover how to access the camera and use that as input for a CoreML model. A CoreML model is a pre-trained Machine Learning model that can be read by iOS.
Duration
- 15–45 minutes 🕧
Requirements
- Xcode 9 or above
- iOS 11 device compatible. Ideally one that can run ARKit (iPhone 6s, 6s+, 7, 7+, 8, 8+, X, SE. iPad 2017, iPad Pro (any))
Tutorial List
Below you’ll find a curated list of tutorials on how to load a CoreML model onto your iPhone.
( Author note: As there were plenty of tutorials covering this step, I’ve opted not to write my own for now. I’m too excited getting the next section ready where we’ll be training your own CoreML model! )
Skim through these tutorials, pick one, and try it out.
- “VisionKit and CoreML” — Chris Greening
- “Building an object recognizer iOS app using CoreML, Vision and Swift” — George Jose
- “How to build an image recognition iOS app with Apple’s CoreML and Vision APIs” — Mark Mansur
- “CoreML: A Quick Walkthrough” — Nick Bourdako
- “Core ML Tutorial: Create a Simple Machine Learning App — Image Classification” — Brian Advent’s Youtube
- “Introduction to Core ML: Building a Simple Image Recognition App” — Sai Kambampati (AppCoda)
- “Core Machine Learning for iOS Developers”– Ayoub Khayati
If you’re overwhelmed, I would recommend Chris Greening’s tutorial on CoreML.
Image access on iOS
There are 3 popular ways of feeding an image into CoreML:
- ARKit — Simple to set up and usable with Augmented Reality. Currently suitable for small models (under 200 MBs), as larger models can cause ‘hiccups’ to the camera feed. These hiccups may disappear in the future as Apple continues to tweak ARKit.
- AVFoundation — The tried and true method of accessing the camera in realtime. This is suitable for models of any complexity. Requires many lines to set up.
- UIImagePicker — Allows a user to choose a photo from their photo library. Not suitable for real-time applications.
Don’t worry if your tutorial didn’t have ARKit, we’ll be covering it in our next article.
Optional Reading
If you’re new to Machine Learning, I’d recommend checking out Apple’s ML Introduction: WWDC 2017 — Introducing Core ML (31 minutes).
If you’re an intermediate coder (i.e. if you’ve heard of Python, AVFoundation, Tensor Flow), I recommend checking out this video by Alexis Gallagher: “Everything a Swift dev needs to know about Machine Learning” (21 minutes).
Next
The models in the above list of tutorials are generic models that tend to recognize many objects at the expense of accuracy. If we want an application that can accurately recognize our own objects, we’ll need to feed in our own data.
So next, we’re going to create our own CoreML model to classify our own objects. Excellent for recognizing pets 🐁🐠🐕🐘, plants 🌸🌻🌺🌵 or to See Food 🌭.