Creating your own custom machine learning model for CoreML image recognition using TuriCreate
This is part 1 of our 2 part series on how to use our ReactNative realtime image recognition component.
In this article we’re going to create a new machine learning model from scratch using TuriCreate and then export it into CoreML format so that it can be used by iOS apps.
To keep things simple we’re going to train our model to recognise hot dogs or not hot dogs (as seen in Silicon Valley!). That means that, given an image (or live camera view) our model will classify the image into one of two types — either a hot dog or not a hot dog. This is a really simple way to get started, but will also show you the principles needed to extend your model further to recognise more types of image.
Getting the training data
When you create a new ML Model, the first thing you need to do is build up a set of training images. For what we’re doing we need two sets of images:
- Hot dogs 🌭 — loads of images of hot dogs in different settings, lighting conditions, etc.
- Not hot dogs — this is an interesting one. In order to classify something as not being a hot dog we need to build a dataset of things that aren’t hot dogs. This is basically everything else in the world that isn’t a hot dog, but we don’t have time to train on everything in the world, so for now, we’ll just use images of people 👩💻, furniture (e.g. tables , chairs) and other types of food 🥑
There’s lots of different ways you could get hold of these images — googling them, searching on Flickr, taking your own photos, etc. All of these take a fair amount of time and effort, but luckily there’s a website called ImageNet which already contains a large number of image data sets that can be downloaded.
Head over to ImageNet now and search for “hot dog”…
You’ll get one “synset” result. Click on the link and you’ll see the following screen…
Check that the results do indeed show hot dogs, and if you’re happy click on the Downloads tab, and then on URLs. After a few seconds, a page will open with a long list of URLs — copy the URL of this page, and we’ll use it in the next step to download all of the images.
Downloading the image files
It would be really tedious to have to download these image files manually, so we’ve build a node script that will do this for you, and dump them in a training data folder. To get this script, download our coreml-training-tools project from the link below and extract it to a folder on your computer.
Once you have the folder extracted, open up a terminal window and browse to it. Then run the following commands to install the node modules.
cd download-images
npm install
After everything is installed, you can run the image downloader with the following command, where the first parameter is the URL to the ImageNet synset (from our previous step), and the second parameter is the classification/label that you want these images to have, e.g. hotdog:
node download-imagenet-files.js http://www.image-net.org/api/text/imagenet.synset.geturls?wnid=n07697537 hotdog
The images will start to download and will take a couple of minutes to complete. Eventually you’ll get an “All Done ✅” message and browsing to training_data/hotdog will show you all of the images that have been downloaded.
Repeat for not-hotdogs
Now we need to repeat the above steps for not-hotdogs. As mentioned earlier we’ll use synsets of people, furniture and food for this training set. To get the list of image URLs you can search on the ImageNet site, or cheat by using the URLs we’ve already found below:
node download-imagenet-files.js http://www.image-net.org/api/text/imagenet.synset.geturls?wnid=n07942152 not-hotdog
node download-imagenet-files.js http://www.image-net.org/api/text/imagenet.synset.geturls?wnid=n03842156 not-hotdog
node download-imagenet-files.js http://www.image-net.org/api/text/imagenet.synset.geturls?wnid=n00021265 not-hotdog
Time to train
So now we have all of our training images, it is time to get started building the model with TuriCreate. TuriCreate was recently open sourced by Apple, and essentially it makes it really easy to create decent ML models using not very much data.
The easiest way I’ve found to install TuriCreate is via Anaconda (as detailed on the github page above). Note: there seems to be an issue at the moment with CoreML and the homebrew installation of Python. So if you are getting odd errors, its worth going through the install steps again with Anaconda.
Once you have it installed, and are running the virtual environment in a terminal window, browse to the coreml-training-tools folder you downloaded earlier and then the train-model subfolder. From there, run the following command:
python convert-to-coreml.py
This uses the training data you have downloaded in download-images/training_data and creates an ML model file. It takes a little while to run (the time varies depending on how much data you have), but once it has completed you will have a brand new shiny model file called MyClassifier.mlmodel. This file can be dragged straight into XCode and used in your Swift, ObjectiveC or ReactNative projects.
Check out part 2 of our tutorial for how to use this model in our ReactNative component.
