What is Edge computing?
With the widespread success of machine learning research in academia and industry, many businesses are looking to apply ML in day to day tasks. Some applications require real time insights and for these ML models are being deployed on edge. Usually onsite sensors collect the data and send it to the cloud for heavy computation and control actions are then sent back. But this sense -> communicate -> compute -> communicate -> control cycle is not feasible for critical applications because of the possibility of network lag, data leakage and slower overall response. Therefore the emphasis is moving towards doing it all in one place at the edge.
In this blog, we are looking at using the Nutanix Xi IoT platform to deploy a model that can detect cars accurately. This model can be deployed on a traffic light camera to count the number of cars crossing the intersection every day. City officials could use this data in planning infrastructure investments, monitoring safety of intersection, and much more..
Xi IoT platform
Deploying machine learning models on edge devices, analyzing data coming through the sensors and taking right control actions is a challenging task. Firstly it requires expertise from multiple domains for instance machine learning and embedded systems. Secondly different edge devices come with different set of development tools making data format conflicts quite common. Nutanix recently launched their Xi IoT platform. The Xi IoT platform provides a low code framework where edge and cloud computing come together in the most simplified manner so that developers can focus on business logic rather than troubleshooting edge and cloud development or communication issues.
The Nutanix Xi IoT platform is a 100% software-defined solution that delivers local computing, machine learning and intelligence for your IoT edge devices, converging the edge and your choice of cloud into one seamless, delightful application development platform. Nutanix Xi IoT eliminates complexity, accelerates deployment and elevates developers to focus on the business logic powering IoT applications and services.
Running pretrained object detection model on youtube data set
Nutanix offers a free trial version of Xi IoT where one can get an object detection model up and running very quickly and swiftly. Click here to get started with this trial. The trial allows the user to test inference using a pre trained SSD Mobilenet model on two input data sources — YouTube-8m video dataset or use personal sensor such as a smart phone camera to capture and feed the video in real-time. The detection results can be viewed via http live stream in the user dashboard as well as on the phone. Using object detection from your phone just requires downloading and installing the Xi IoT Sensor app from Google or Apple app store, scan a QRcode from the tutorials page and open up the phone camera for inference.
Using their platform to do this was a very cool experience! It is amazing how all this can be done in a few clicks. The main steps I followed were:
- After logging in to my Xi IoT trial account, from the “Quick start” tab of the management console, I chose “Connect your phone” option. Downloaded the Xi IoT Sensor app from playstore and connected my phone to the XI IOT platform by simply scanning a QRcode.
- Then from the Xi IoT management console, I chose Apps and Data > Data Pipelines. And started the phone object detection data pipeline.
- Next, I could capture the video through my phone’s camera that was connected to Xi IoT with phone object detection pipeline running. That was it, I could see amazing object detection running on my phone.
The platform provides an excellent online tutorial with detailed steps for doing object detection either with your phone or with any video of your choice.
Personally as a developer this is very exciting as it allows me to test on my own camera stream or online videos quickly with just a few clicks. The quality of results was quite impressive too. See gif below.
As a developer, I also want to be able to load my custom trained tensorflow object detection models and run inference on them online real-time. The platform makes it easy to do this too :)
Deploying custom model
We really liked that the platform isn’t just limited to the default object detection model but you can upload your own custom models.
Apps and Data Options
Apart from default Applications and Data Pipelines, there are options to create one’s own Functions and upload ML Models. All you have to do is upload your own model with zipped saved_model.pb and variables, name your model, assign it a version number and then modify a few lines in the scripts inside Functions.
In Functions, there is a script named ml_objectdetect_func-python which is being used by ai-inference-service-demo pipeline. You just have to make a copy of this script using the clone option, name this new script and modify the following lines:
labels = [“car”]model_name = “car_model”model_version = 1
Assuming you had uploaded a model that detects cars. And you uploaded the model with name ‘car_model’ with version ‘1’. Then you can just edit the ai_inference_service-demo to use the function named ‘your_function’ instead of ml_objectdetect_func-python. Again, you can find detailed step-by-step official instructions to do that here.
We deployed a custom cars model and it was very cool to run it on a mobile phone. See our results below:
Nutanix Xi IoT platform is a powerful tool to deploy your custom object detection models and run edge inference on them. You can sign up for a free trial here and contact Nutanix at email@example.com for any questions.