Here at Skafos, our main goal is to make it easy to deliver your machine learning models to the edge, whatever your training environment. We are so fortunate that we have a number of excellent resources to choose from, including IBM Watson. In particular, IBM Watson supports the training of custom visual recognition models without needing to write any code, and these models are automatically generated in Core ML format, which makes integration into iOS apps exceptionally easy.
There are a number of great tutorials of how to do this, in particular this one by Nick Bourdakos. I also had the opportunity to attend a Lab at AltConf 2019 in San Jose given by Marek Sadowski, and he walked us through this tutorial. In both of these examples, Nick and Marek integrate their models into an iOS app.
In this article, I’m going to start with what they’ve done, but show you how to use Skafos to update, deliver, and manage your models. What if you’ve used IBM Watson to train your model, but you need to deliver it into an app you already have? How do you incorporate improved models into your app without having to re-submit to the app store? What if you need to rollback to a previous version? Stick around, because I am going to walk you through it.
Before beginning this tutorial, I’m assuming that you’ve already done the following:
- Worked through either of the linked tutorials above to generate an Image Classification (Visual Recognition, in IBM parlance) Core ML model.
- Signed up for a Skafos account, logged into the dashboard, and clicked the Try Example App link to create an example Image Classification iOS app.
- Walked through the Skafos integration guide to get this app on your phone.
Uploading your model to Skafos
I’m going to start with the model I built during Marek’s tutorial at AltConf. It identifies different types of connectors (HDMI, Thunderbolt, USB, VGA). The first thing I’m going to do is download it so I can transfer it to Skafos. You should do the same.
After downloading, I renamed the model from
ImageClassifier.mlmodel, and zipped it up using the standard Apple compression functionality. The file rename is needed to make sure that the Core ML model name is consistent with the model naming conventions I already have in my Skafos app. You should do the same if you are using our ImageClassification iOS template.
Next, log into your Skafos dashboard, click the card for the Image Classification app you already created, and click the ImageClassifier model card. It will take you to your model management dashboard for this app, which looks like this:
Now, you’ll want to focus on the Model Upload section and upload
ImageClassifier.mlmodel.zip. Adding a description is super helpful as well.
After uploading, your Model Versions list should now be incremented by one, with the model you just added included.
Delivering a New Model to your iOS App
My connector model is now associated with my Image Classification app, and yours should now be too. But how do you deliver this model to your phone? By the deploy button of course!
Before doing that, let’s quickly use the example iOS app you already installed on your phone. In the image above, there is a Dev tag next to model version 1. This model identifies different types of poison ivy. I chose to deploy this model with my app, and so everything I photograph is identified as a type of poison ivy. Let’s try this with my favorite coffee cup. (You can do something similar with any object nearby.)
If you are confused about why my coffee cup is being identified as Eastern Poison Ivy, check out point #2 in a recent article I wrote about this topic. Also, if you’ve walked through our example Image Classification app, and not deployed a model, the app will pull our default classifier, which identifies cats and dogs.
Now, let’s replace this poison ivy model with the connector model I’ve built, and see how things change.
After clicking Deploy, you will want to select Dev as your environment and click the Confirm & Deploy button. Prod is for apps that have already been through Apple’s App Store Review process. The mechanism of delivery is exactly the same, but you probably don’t want to go through the review process for this example.
Now that you’ve delivered this model, the location of the Dev tag will be updated in the Model Versions table.
Refresh the app on your phone, use the same photo, and see what happens.
Want to roll back to the previous version of your model? Add a new model? Build it in IBM Watson as you did previously, upload to Skafos, and once it shows up in the Model Versions table, Deploy to your heart’s content. This process can be repeated as many times as needed. The process of building good machine learning models is inherently iterative, and we’ve built Skafos to support that.
That’s it. I hope that by now, you are getting a sense that by pairing IBM Watson’s model training capabilities and Skafos’s model delivery management, you can build a complete, reliable, and reusable mobile machine learning pipeline. Machine learning is successful only when you can iterate quickly, and these two platforms together allow you to do just that. Please reach out and let us know how you are building smarter apps with better user experience. We’d love to hear about it!