Effective TensorFlow for Beginners
The last talk I gave before the 2020 global crisis was at an Artificial Intelligence meetup in Karachi organized by Google Developers Group (GDG Kolachi). The attendees comprised of professionals and students both, there were developers eager to grasp the concept of Machine Learning and looking for ways to integrate it into their applications, students present wanted to explore the areas which were the ‘buzz word’ and lastly there were practicing ML experts who either wanted a quick recap or hadn’t worked on TensorFlow framework. Keeping the audience in mind, I drafted the talk in a way that everyone should benefit from it in some way or the other.
As I am a strong advocate of #sharingknowledge and the #opensource community, This blog highlights the important pointers and take away messages from the talk to benefit a greater number of people.
Object Detection and its Applications
Let us first look at some of the applications of deep learning models that we can build using TensorFlow framework. The figure above shows results from vehicle detection model if you are wondering about how such models are developed and trained then the basic pipeline of training any customized object detection model is as follows;
- Collect datasets for desired objects (the already trained models have a number of objects already present in the list — but you might still want to fine tune* them on your native datasets)
- Label the datasets using a labeling tool such as labelImg. (If you are training a model from scratch then 1000+ images of an object are recommended whereas for transfer learning or fine-tuning 300–400 can be used as well.)
- Build model architecture from scratch or use an already trained model for transfer learning*
- Train the model using your customized datasets
- Test model on unseen data to evaluate results
Once you have your object detection model ready, there are numerous applications that you can build on top of it where object detection can be the baseline model for your application. Some examples are;
1. Security App — identifying prohibited or abandoned items in a facility
2. Brand Visibility App — check your brand views using logo detection
Similarly, the short clip below shows real-time machine translation. This is an example of Optical Character Recognition where the text is generated from video/ image. Other OCR use-cases can be Automatic License Plate Recognition (ALPR), Information extraction from images of cards and documents.
Unprecedented Resource Availability
Previously, it was a hassle to acquire or build the datasets, but now with the opensource community, it has become quite trivial. Just recently, Google published 25 million free datasets (Try at datasets by Google). Similarly, a storage device in 1956 could only contain 2 digital photos in contrast to the 12 million digital photos in a single drive in 2020, hence storage these days is not a problem anymore. Subsequent to data and storage comes to the processing power ranging from CPU to GPU and now even TPUs are available. Given the above three resources, one can easily develop and train machine learning and deep learning models. But another question that arises is how to deploy and integrate these models into usable applications on Android, iOS, web platforms or on edge computing devices? That is where TensorFlow comes in!! It serves as the bridge between your machine learning application and the device.
TensorFlow is an opensource framework by Google with excellent community support available across multiple platforms. TensorFlow itself is quite a user-friendly framework with descriptive tutorials and resources available in abundance, it also has high-level interfaces built on top of it such as Keras, Ludwig, Sonnet etc. It is fast, flexible and comes with a model zoo that comprises of numerous pre-trained models that can be used as-is in your applications or fine-tuned on your custom data. Along with that, it comes with certain prebuilt APIs in multiple languages. It has become the industry standard for deep learning applications as a lot of big tech companies have adopted TensorFlow in their applications. TensorFlow is compatible with CPU, GPU, mobile processors and is supported by cloud platforms as well. Hence, one can rapidly learn and practice.
Let us take a quick look at how TensorFlow Lite works in minimal three steps,
- Pick a pre-trained TensorFlow model from the model zoo
- Run the model on Google Colab through TensorFlow object detection API
- Convert the TensorFlow model into a compatible compressed file using the TF Lite converter
Let’s Do Some Coding
For a hands-on experience, let us work with a pre-trained model available in Tensorflow Object Detection API. We will run it using Google Colab, which facilitates you to run your code in google cloud servers.
To start with coding, open a new tab and go to https://colab.research.google.com . From the dialog box select New Notebook. This will open a Jupyter Notebook for you hosted by google. The code snippets explained below can be found at my git repo.
The commands above are used to set up Tensorflow Object Detection API on your notebook environment.
We import the necessary libraries. The good thing about google colab is you don’t have to worry about libraries and their installation, they come by default.
We will be using the SSD Mobilenet trained on COCO data for object detection. It has 90 object classes ranging from a person, cat, dog to car, bicycle etc.
Running the model we can see that we are able to classify dog in test images with an accuracy greater than 90%. Once the model is ready and up to the mark, we can now convert to TFLite. There are multiple methods of converting a model into a compressed model using the TFLite converter but since we are using the pre-trained object detection model by Tensorflow its converted version is easily available from here. This TFlite model can then be deployed on your mobile application or edge devices easily.
Hence, setting up the Object Detection API in Google Colab is a trivial task and can be done using the steps defined above. On top of this you can built numerous applications and projects (THINK BIG, THINK WILD). I also intend to write subsequent articles on training and deploying custom models, so stay tuned for that.