Machine Learning and iOS
The whole area of Machine Learning has come a long way in the last years and so have the different ways we can interact with it as iOS developers. In this post I wanna cover what Apple has created for us developers to use Machine Learning in our apps, since Accelerate all the way to TuriCreate.
I think there's a lot of possibilities to be explored in the intersection between iOS and Machine Learning. Maybe this post will inspire you or help you find a nice way to use Machine Learning in your apps.
Accelerate and Metal
2017 was an amazing year for those interested in the intersection between Machine Learning and iOS development. A few weeks before the WWDC17 I was working in a project where we had to use convolutional neural networks in order to classify images. My colleagues and I were wondering what we would have to do if we wanted to make our model work inside an iPhone and we came across Accelerate and Metal.
Using this approach, what you have to do is train your model using your favorite framework and, once you have your model, that's where Accelerate or Metal come into play. You need to load the resulting weights from your model to Xcode and use them to create each layer of your network.
There's an example here on how you can use a neural network that recognizes handwritten digits using metal. The project is an app with one view controller where you can (1) test the accuracy of the selected network in the MNIST dataset; (2) draw a digit and check what is the recognized digit; and (3) you can select whether you wanna use a single layer network or a deep network.
If you download the project you will see that the weights and biases for the layers are in stored as .dat files. Then, you would need to load those data into the layers.
In order to make a prediction for some given input data, you’d have to pass the data through each layer until the softmax layer.
Then all you have to do is get the label in a "human-presentable" way from the softmax layer and present it to the user.
It looked like a lot of work in order to add a simple Neural Network to an app and we felt that apple should present some enhancements to it in the WWDC, since “Machine Learning” was everywhere — and, by the way, Google had just announced TensorFlow Lite, which allows Android developers to use models built with tensor flow in their apps.
We couldn't wait for the WWDC!
Core ML
The WWDC keynote was awesome! And, particularly for me, the best part was the announcement of Core ML, which was way beyond what I was expecting. That same night, my friend Aleph and I, started trying to figure out how to use Core ML and that’s when we wrote this.
Well, if you wanna use Machine Learning models in your apps but Accelerate and Metal seemed like too much, you might wanna get to know this one.
With Core ML you can get a model trained in one of many different machine learning frameworks such as Keras, TensorFlow or Caffe. Then, once you have your model, you can use coremltools to convert your model to the .mlmodel format, which can be imported into Xcode.
When you import a model to Xcode, you will have a Class automatically generated for you with the same name as your model and you can simply call the method prediction(input) and it just works!
Of course, you're gonna encounter some difficulties in order to import it to Xcode or when preparing your data to input into your model, but the difference from the ammount of work you'd have to do with Accelerate or Metal is huge.
And for those who wanna start and don't really know how to create a model, you can find some pre-trained models here and here. The first one is Apple's official page with some famous models already exported to .mlmodel and the second is a GitHub repo with many models created by developers all over the world with which you can also contribute. Check it out!
Now, if none of the models in those lists suit you and you have the data to train but cannot create a model yourself just now, the next section is for you!
(I'm starting to sound a bit like a bad advertiser, but I'm really excited about that next one!)
Turi Create
Months went by and everything was great. Then one afternoon, in the last days of our spring here in Brazil — almost winter in the north — I was checking out my GitHub updates and someone had starred this repo called Turi Create.
If you think this is all good but Machine Learning is just way to hard, take a look at this one!
In their own words:
Turi Create simplifies the development of custom machine learning models. You don’t have to be a machine learning expert to add recommendations, object detection, image classification, image similarity or activity classification to your app.
After the launch of Core ML many iOS developers started studying Machine Learning, but it is in fact a huge area and it takes a lot of study and practice in order to be able to create your own models.
So what if you could create a baseline model that you could quickly start using in your iOS apps and validate your idea until you're able to create better models or find some data scientist for your team? That's what Turi Create allows you to do.
With few lines of code you can have your model (a decent one) created, trained and exported to the .mlmodel format.
Turi Create still have some problems if you need specific models, but the good news is you can collaborate with the team finding bugs, creating issues and even helping fix them.
Personally, I think there's an enormous world of possibilities in the intersection between Machine Learning and iOS and there's never been a better time to be a developer who likes working with both those areas.
2017 has been a great year for us and I hope we can create a lot of amazing things using those technologies in 2018 and the years to come. I’m really excited about it, so let me know what you're working on!
That's it! If you have any questions, comments or suggestions, you can find me on Twitter.
Thank you for reading! 🙃
