How did I used Intel Movidius Neural Compute Stick from scratch

“We can do for free what some can’t do for money”

My MNCS posed on Rayan’s desk (my first cousin once removed) 👊

I can’t forget my excitement in 2016 when Movidius, the company based in San Mateo, California that designs specialized low-power processor for computer vision before being acquired by Intel in September 2016, announced the soon-availability of the Neural Compute Stick that promised low-power computer vision capabilities for end to end devices. I waited patiently, until the summer 2017 when the first few hundreds of sticks made available for the general audience. Then, actually I had to wait until 2018, when my company delivery manager, managed to order me this one.

Nowadays, the CPU (central processing unit) has often been called the brains of the PC. But increasingly, that brain is being enhanced by another part of the PC, the GPU (graphics processing unit), which is its soul. All PCs have chips that render the display images to monitors. The need for AI accelerator hardware has boosted the innovation and production of some new processing units, the TPU (tensor processing unit) which is an AI accelerator application-specific integrated circuit (ASIC) developed by Google specifically for neural network machine learning(2016) and the VPU ( Vision processing unit) which is high performance and advanced software for on-device vision AI (MNCS is one of the most known VPU, ultra-low power, low cost and high performance).

What is the Intel MNCS?

We can think of the MNCS like a USB really powered GPU, although it is not a GPU and it can only be used for prediction/inference, not appropriate for training. I would classify the MNCS as a second processor with one purpose: running vision neural network calculations. In this blog, we’ll be using the NCS for image classification and object detection.

The MNCS should not be used for training DNN models rather it is designed for running deployable models. Since the device is meant to be used on single board computers such as Raspberry Pi, the power draw is meant to be minimal, even it’s ultra-low power which making it inappropriate for actually training DNN.

It’s really low cost. You can buy the device on Amazon or at any of the retailers listed on Intel’s site.

Under the hood of the NCS is a Myriad 2 processor capable of 80–150 GFLOPS performance. This processor is high performance, vision accelerator and it consumes only 1W of power (for example, the Raspberry Pi consumes about ~1.2W).

OS

Intel Movidius is an interesting hardware that could work on embedded system like Raspberry Pi that will enable neural network computing. Even thought is only supported under Linux, we could always use Virtual Box, set a Ubuntu 16.x (the best for us: 16.04)virtual machine to get it to work on Mac or Windows platform.

Frameworks

MNCS also support both Caffe and TensorFlow models and the typical workflow is as follow

The typical workflow for development with the NCSDK, please visit this for more informations

So the MNCS has two general usages:

• Profiling, tuning, and compiling a DNN model on a development computer with the tools provided in the NCS SDK.
• Prototyping a user application on a development computer, which accesses the neural compute device hardware to accelerate DNN inferences using the NCS API.

In general, the workflow is:

1. Use a pre-trained TensorFlow/Caffe model or train a network with Tensorflow/Caffe on Ubuntu 16.
2. Use the NCS SDK toolchain to generate a graph file.
3. Deploy the graph file and NCS to your single board computer running a the Ubuntu colors of Linux.
4. Use the MNCS API to send the graph file to the MNCS and request predictions on images. Process the prediction results and take an action based on the results.

Installation

If you are not using the ubunto Linux OS, we recommand you to follow this steps before deploying an MNCS workflow:

Setting up a virtual machine:

The first step is getting a virtual machine(VM) up and running. Although there are a number of different VM software options, Virtual Box is a freely available one, simple to configure and use. Alternatives such as VMWare may provide better performance if the VM is intended to be used as a primary workstation.

1. Download Virtual Box.
2. Install VB using the installer.
3. Download Virtual Box Extension Pack.
4. Install Virtual Box Extension Pack using the installer.
5. Download Ubuntu 16.04 64 bit ISO image.
6. Create a new virtual machine.
7. Load Ubuntu 16.04 image as optical disk on the newly created virtual machine.
8. Start the virtual machine.
9. Follow the steps to install Ubuntu on the virtual machine.

Feel free to adjust the memory size and the virtual hard disk allocation as you want, but just keep in mind that over-allocating resources will result in poor performance on the host operating system.

Setting up your ubuntu 16.04 machine:

Before installing our MNCS SDK and running some examples up, some dependencies are required to ensure that the development environment is ready and the necessary tools are available. This entails updating Ubuntu and making sure you have Python, PIP, and Git to clone code repositories.

Let’s update the system:

sudo apt-get update && sudo apt-get upgrade

Then let’s install a bunch of packages:

sudo apt-get install -y libusb-1.0–0-dev libprotobuf-dev
sudo apt-get install -y libleveldb-dev libsnappy-dev
sudo apt-get install -y libhdf5-serial-dev protobuf-compiler
sudo apt-get install -y git-all
sudo apt-get install -y libatlas-base-dev git automake
sudo apt-get install -y python3-dev python-pip python3-pip
sudo apt-get install -y python3-pillow python3-yaml python3-pygraphviz
sudo apt-get install -y python3-h5py python3-nose python3-lxml
sudo apt-get install -y python3-matplotlib python3-numpy
sudo apt-get install -y python3-skimage python3-scipy
sudo apt-get install -y python3-six python3-networkx
pip install scikit-image

Notice that we’ve not installed libopencv-dev from the Debian repositories. If you did it, hope that this is the last time as well. In fact, Installing OpenCV via apt-get installs an older version of OpenCV and does not install the full version of OpenCV in addition that it does not take advantage of various system operations. Again, I do not recommend this method to installing OpenCV. Additionally, you can see we’re installing a whole bunch of packages that we generally prefer to manage inside Python virtual environments with pip.

Since we’re using OpenCV and Python, we’ll need the python-opencv bindings. The installation instructions on the Movidius blog don’t include this tool. You may install the python-opencv bindings by entering the following command :

sudo apt-get install -y python-opencv

Plug your MNCS and make sure the Stick is recognized

Now, it’s time for the most important part. This involves ensuring that the our MNCS is recognized.

1. Plug the MNCS stick into a USB port.
2. Use the lsusb command in Console to determine if it is recognised by Ubuntu. You should see the MNCS stick in the list of USB devices. ID as 03e7(usb 2) ,ID as 040e(usb 3) .
3. If you are using a VM and it is not recognised, follow the below instructions:
4. Navigate to the settings of the VM and choose Ports > USB.
5. Add a new filter for USB 2 providing just the ID as 03e7/ add a new filter for USB 3 providing just the ID as 040e
6. Start the Ubuntu VM.
7. Use the lsusb command to list USB devices, and the MNCS stick should now be recognized.

Installing the Neural Compute Software Development Kit

The MNCS SDK is required to interact with the Movidius stick. From there, let’s navigate to our desktop in Console and clone the NCSDK:

cd ~
cd Desktop
git clone https://github.com/movidius/ncsdk

In the NCSDK directory we’ll use the Makefile to install SDK and build the examples

cd ~/Desktop/ncsdk/
make
sudo make install
sudo make examples

Testing the NCSDK installation, Building and Running examples

Let’s test the installation by using code from the NC App Zoo. Be sure that the MNCS is plugged. The Neural Compute App Zoo is a repository of examples that demonstrate how the NC SDK and Movidius stick can be used to train and process neural network graphs more efficiently than typical CPUs.

cd ~/Desktop/
git clone https://github.com/movidius/ncappzoo
cd ncappzoo
sudo make install 
cd caffe && make
cd ~/Desktop/ncappzoo/apps/hello_ncs_py
python hello_ncs.py

You should see this result

if the exact output is as above, congratulation your MNCS is actually working.

First example (Image classification)

Finally, we get to run some examples and see the Movidius in action. The first example we will be looking at is image classification. Instead of creating our own model which will take countless of hours of data collection, data processing, and training, we will simply use GoogLeNet — a well trained model for image classification by Google. The graph engine used to process the model will be Caffe. Caffe is a widely used machine-vision framework which excels at image-related tasks.

cd ..
cd apps/image-classifier
python image-classifier.py

You should see results of the basic image classifier.

Second example

The second example we will be looking at is object detection

cd ..
cd apps/object-detector
python object-detector.py

Success! You should see results of the basic object detector recognizing the chair

Good and Bad

+ :

• a lovely piece of “AI usb hardware”
• delivers speed as it’s promised (80.4 ms for object detection in our basic machine)
• Raspberry PI support
• Both Tensorflow and Caffe support
• open source repos

— :

• need Ubuntu host to compile neural network (no Windows, no OSX, although a VM could do)
• not appropriate for training
• obviously it supports only the AI vision part, no natural language processing, neither text nor voice( chatbot ..).

Go, We Can

Although the MNCS only supports Raspbian and Ubuntu at the moment, it is possible to get it running on your platform Windows or OS using a VM. May be soon, we will be seeing AI on edge devices such as drones, home automation surveillance cameras, and other IoT devices via purpose specific hardware solutions such as the MNCS. In fact, let’s hope this will be for the good reason

Summary

Today we explored Intel’s Movidius Neural Compute Stick. Our goal was expose you to this new deep learning device. We also demonstrated how to use the NCS workflow and API.

In general, the NCS workflow involes:

1. Training a network with Tensorflow or Caffe using a machine running Ubuntu. (or just use a pre-trained model like we did)
2. Using the NCS SDK to generate a graph file.
3. Deploying the graph file and NCS to your PC running Ubuntu 16.04.
4. Performing inference, image classification, object detection, etc.

Today, we skipped Steps 1 and 2. Instead, NCAPPZOO provide graph files which you can begin using immediately.

Then, we runned a classification benchmarking Python script and analyzed the results which demonstrate a significant 5x speedup on our machine( graphic card is Nvidia Geforce 920M).

If you are interested in pushing the limits of the Intel Movidius Neural Compute Stick. Intel is sponsoring a competition on TopCoder.

So explore the examples and build your own AI, share your great work, and help the AI community to mould the future.

Hope you enjoyed this introductory post on Intel’s new MNCS!