Meir Rosendorff
Sep 9 · 5 min read

Machine Learning in swift why, what, how

source https://twitter.com/TensorFlow/status/1103393545866657792

Imagine a powerful, well documented easy to use language available to the masses that easily allows one to learn and implement machine learning solutions. It already exists. For many years Python has been the go to language for machine learning students and enthusiasts. This is because of its low barrier of entry, ease of use and powerful ML frameworks; one of the most famous of which is TensorFlow.

Is this going to change?

Now imagine another language that is compiled, type safe, has good memory management, great performance and has all the power that Python has but in a safer, more scalable package? This was Google’s vision when it decided it would develop a new, more powerful TensorFlow framework, a framework that would take machine learning to the next level. They chose to do this in Swift.

Swift is a fairly new language developed by Apple, who for the past few years has been developing their new framework, Swift for TensorFlow. You can check out Google’s full set of reasons and their github project’s here.

I’m an iOS Developer and when I found out about this all I thought is “what could be more cool than building, training and deploying an ML model totally in Swift”. If you aren’t an iOS dev don’t run too quickly because this may very well be the future of ML, or will it? That is what I set out to discover.


The Journey

Setup and Installation

The first step in my Swift for TensorFlow journey was the setup and installation. The Developers recommend using Google’s Colaboratory because their support and development is focused in that direction, but I thought to myself if I am going for the future of ML I want to write my programs in a real IDE and so followed their instructions to install it on Xcode. To cut a long story short after two frustrating days I eventually got it installed and running in Xcode 11, you are welcome to check out my installation instructions and give it a go yourself.

Python Interoperability

One of the greatest struggles with attempting ML in languages other than Python is that a huge effort has gone into building powerful data manipulation libraries for Python. The Developers of Swift for TensorFlow decided that rather than writing off these libraries or rebuilding them, they would create an interface for calling python libraries and functions ‘natively’ in Swift. I must be honest this is really cool! You can check out the full docs but let me show you some neat examples here .

The above shows an example of using the python io library in swift to read in a file line by line, we then convert the line PythonObject to a swift String and call native swift functions on it from that point. Heres another nice example:

In the above code we use the python module numpy to create a 3D array of ones, we then divide each entry by half. We can even get the shape of the array, convert that to a swift tuple and use it normally as we would any tuple.

Mind Blown
Mind Blown
Source https://knowyourmeme.com/memes/mind-blown

This is all awesome. You can now easily harness all the power of Python in your Swift code in a intuitive and concise way. This is the dream isn’t it? Theres a catch though, this is not type safe. What this means is you can attempt to call members of an object that don’t exist, such as np.someFunctionThatDoesNotExist(), or perform an invalid operation and it will still compile but will crash at runtime. This feels like we are going backwards and not achieving what we set out to achieve.

Training a Model

Ok I’ve spoken enough about cool features but let's get to the crux of it, can you train and deploy an ML model using swift?

The first model I trained was quite straight forward, I followed the online tutorial and got it running without too much issue. But this didn’t satisfy me, I wanted something juicier that would look more impressive in an App. So I found an online dataset of American Sign Language Letters and set out to train a model that would be able to translate hand signs live.

After about a week of work I managed to get a functioning model that was getting pretty impressive accuracy on my testing dataset. At this point I was excited and wanted to get this into an App. This is when I ran into my first major issue. I could not find a way to save my model. Unfortunately this feature has not yet been developed. This is not small hiccup, the ability to save and reuse work you have done is a cornerstone to any functional framework. This was one of the biggest red flags for me that the framework just isn’t quite there yet.

I ended up having to hack my way through it and eventually using the Python Interoperability to import the Python’s TensorFlow module, copied my weights over to it after training and saved it from there. I was then able to convert this model to a coreML model (iOS ML model format) and use it in my app.

The below video is of the functioning app, the correct classification for the hand sign is the letter K.

Please take a look at the app on my github to see exactly how I did it, or take a look at a more simple example that I made for a workshop here.


Conclusion

So is it possible, are we now able to build, train and deploy ML models directly from Swift? Did Google successfully port TensorFlow over to a type safe, compiled language and begin the future of ML?

In answer to the first question it is possible to do but it is not yet commercially viable as there are still massive wrinkles that need to be ironed out before the framework reaches a point that anyone would be willing to use in production.

I think Google has made great strides towards realising their dream and I do feel that ML will head in this direction, but not anytime soon. Python is a great language but it does have many drawbacks. There is no doubt that it will still be the king in the ML arena for many years to come but Swift is catching up. I believe before we know it Swift for Tensorflow will have developed enough to use for a lot more than a just fun challenge.

Do you think all this work Google is putting in is necessary or is Python good enough ?

Have any of you tried out the new Swift for TensorFlow framework and found your experience different to mine?

Let me know your thoughts.

DVT Software Engineering

Making an impact in Software Engineering

Meir Rosendorff

Written by

iOS Developer using swift and ObjC, Passionate about learning. Currently working on my ML skills particularly in Computer Vision and Reinforcement Learning.

DVT Software Engineering

Making an impact in Software Engineering

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade