An Ode to the Terminal: The (Almost) Perfect Tool
Bringing Deep Learning to the Terminal with Node and Tensorflow.js
One of my favorite things about my work as a developer advocate is that I have the opportunity to work with lots of different technologies across all types of projects. This means lots of different tools and workflows, but there’s one tool I use constantly throughout the day for almost everything — the Terminal. There are plenty of options out there depending on what type of environment you’re running (my personal favorite for Mac is iTerm 2) but in my opinion you really can’t go wrong. Nothing feels quite as powerful as entering in commands at the command-line and watching my computer do exactly as I’ve requested, but even my favorite tool does have its limitations.
Conjuring a Solution
Working with large numbers of files can sometimes get a little confusing, especially if they aren’t labelled properly. Digging through log files, raw input data, or something like an SD card full of photos to find the one you want can be an arduous task. For text-based files there’s
cat, a command that I frequently turn to for a quick look into a file’s contents… But what about other types of media, like images and audio files? Sure, you could just open the file and look at it, but that’s not always an option. What if you’re logged into a remote terminal, or using an OS that can’t open these types of media files? Once my curiosity gets going there’s really no stopping it, so I set out to solve this problem. Through the use of TensorFlow.js and an open-source deep learning model from the Model Asset eXchange, I feel like I’ve come up with a solution that works! In addition to showing off
magicat🧙 😺, the tool I created, I want to talk a bit about the tools that made it possible.
Tensorflow.js — Helping Democratize Deep Learning
Until recently, Deep Learning development was a task performed almost exclusively by machine learning engineers, data scientists, and developers with experience in cognitive computing. Using traditional data science and ML tools like Python, numpy, TensorFlow and Keras, these pioneers helped establish pipelines and workflows that would lay the foundation for what was to come. While these tools do their job well, they come with a bit of a learning curve and can be cumbersome at times. Anyone who’s spent time juggling different Python versions and dependencies can attest to how frustrating that can be, and that’s just to get your environment set up. It was clear to me that there had to be a better way.
Before discovering the magic of TensorFlow.js, there were a few tools that really smoothed out the Python dev experience that deserve some recognition. First and foremost — Anaconda. If you’re doing any kind of data science or machine learning with Python, this one’s pretty much a must-have. Installing Anaconda on your machine gives you access to the
conda package manger, which uses pre-built packages for HUGE speedups compared to
pip, but more importantly
conda allows you to manage different “environments” on one machine, which can be activated from the terminal to easily and quickly switch between Python versions or sets of dependencies.
While working with
conda helped make my life easier, for me the perfect solution has arrived in the form of TensorFlow.js. Other developers have already made this discovery and have been singing the praises of TF.js for some time, like Maureen McElaney’s article that talks about the impact this has on offline-first web apps. This is a powerful side effect of hosting models on edge devices rather than in the cloud, and one that I was eager to take advantage of. These benefits allowed me to construct a deep learning powered command line utility that doesn’t rely on any remote service or internet connection.
Putting it All to Use: Building '
magicat' 🧙 😺
Since I knew how I was going to interact with my model, I just had to find one that could identify objects in an image. Luckily, there’s “a place for developers to find and use free and open source deep learning models” called the Model Asset eXchange (MAX) that hosts over 20 different models built for a wide range of tasks. After choosing the Image Segmenter model from MAX, I was able to download and convert the model into a TensorFlow.js-ready format by following the steps outlined by va barbosa in this article and using a tool called
node-canvas had already come up with an excellent solution. Similarly, when I started thinking that it would be cool to show previews of the different objects in the image, a quick search produced the awesome
terminal-image module by Sindre Sorhus which dropped right in to my code. These types of experiences are what drive home the magic of open source software for me. It’s like we’re all building and figuring things out together!
Check it Out
magicat 🧙 😺 on GitHub, or browse the Model Asset eXchange to see what models you can use in your own applications. The list of MAX models is updated periodically and includes models that accept audio samples as input, as well as adversarial cryptographic and generative models.
One more thing… if you enjoyed this post, visit The Data Lab @ CODAIT for more reading on topics like deep learning and open source software.