Unstructured Data Processing with a Raspberry Pi AI Kit — Hailo Edge AI
Unstuctured Data Processing, Raspberry Pi 5, Raspberry Pi AI-Kit, Milvus, Zilliz, Data, Images, Computer Vision, Deep Learning, Python
Detecting, Displaying and Storing Detected Images in Live Camera Streams at the Edge
Even with a small budget, you can start building some Edge AI use cases thanks to the power of devices like the Raspberry Pi 5 with 8GB of RAM and the NVIDIA Jetson Orin Nano. Recently the the Raspberry Pi AI Kit was released for the RPI5 platform, so I had to get one and try it out.
The AI Kit adds a neural network inference accelerator capable of 13 tera-operations per second (TOPS), which is pretty good for $70 US. Attached to this M.2 Hat is the Hailo-8L M.2 Entry-Level Acceleration Module which will give us our AI powers.
In this first demo, I modified one of the provided RPI5 Hailo AI Python examples to do live Image Detection on a web camera then send the detections to a Slack channel and more importantly vectorized with metadata to Milvus.
Running Live From The Raspberry Pi 5
We are using Hailo’s example RPI5 object detection program that we have enhanced to send to Slack, MiNio and Milvus.
So using the example Object Detection program, but first I added some imports for my libraries for Slack, Milvus, S3, TIMM, Sci-Kit Learn, Pytorch and UUID. I also set a few constants to use later. We then connect to our Milvus server and Slack channel and beging our GStreamer loop. I have a time check and on that if something was detected I save the camera frame to a file that I upload to S3 and send to my Slack Channel. Finally I add my vectorized image and important metadata of S3 Path, filename, label and confidence. We get an autogenerated ID for each entry in our collection.
Our images have been uploaded to MinIO:
They have also been sent with our text message to our #reports Slack channel.
Most importantly our metadata and vectors have been uploaded and are already available for super fast search.
We can now start querying our vectors and I show you how with a Jupyter notebook.
Querying the Database and Displaying Images
I have recorded this demo running so you can see what’s going on live.
If you acquire one and want to set it up to replicate my demo, please see the steps at the end of this article.
Demo Packing List
MinIO / S3, Milvus, Slack, Python, Boto3, OpenCV2, Pytorch, Sci-Kit Learn, TIMM, Hailo, YOLOv6n, Object Detection, Raspberry Pi AI Kit, Raspberry Pi 5 with 8GB RAM, logi webcam, resnet34, Torchvision, PyMilvus,
Hailo8L M.2 module, M.2 M-Key Hat, Heat Sink.
Getting Started
So after you have added the hardware (see the videos and links below), you install the libraries, reboot and you should be ready to go.
tspann@five:/opt/demo $
hailortcli fw-control identify
Executing on device: 0000:01:00.0
Identifying board
Control Protocol Version: 2
Firmware Version: 4.17.0 (release,app,extended context switch buffer)
Logger Version: 0
Board Name: Hailo-8
Device Architecture: HAILO8L
Serial Number: HLDDLBB241601635
Part Number: HM21LB1C2LAE
Product Name: HAILO-8L AI ACC M.2 B+M KEY MODULE EXT TMP
tspann@five:/opt/demo $
dmesg | grep -i hailo
[ 3.155152] hailo: Init module. driver version 4.17.0
[ 3.155295] hailo 0000:01:00.0: Probing on: 1e60:2864...
[ 3.155301] hailo 0000:01:00.0: Probing: Allocate memory for device extension, 11600
[ 3.155321] hailo 0000:01:00.0: enabling device (0000 -> 0002)
[ 3.155327] hailo 0000:01:00.0: Probing: Device enabled
[ 3.155350] hailo 0000:01:00.0: Probing: mapped bar 0 - 0000000095e362ea 16384
[ 3.155357] hailo 0000:01:00.0: Probing: mapped bar 2 - 000000005e2b2b7e 4096
[ 3.155362] hailo 0000:01:00.0: Probing: mapped bar 4 - 000000008db50d03 16384
[ 3.155365] hailo 0000:01:00.0: Probing: Force setting max_desc_page_size to 4096 (recommended value is 16384)
[ 3.155375] hailo 0000:01:00.0: Probing: Enabled 64 bit dma
[ 3.155378] hailo 0000:01:00.0: Probing: Using userspace allocated vdma buffers
[ 3.155382] hailo 0000:01:00.0: Disabling ASPM L0s
[ 3.155385] hailo 0000:01:00.0: Successfully disabled ASPM L0s
[ 3.417111] hailo 0000:01:00.0: Firmware was loaded successfully
[ 3.427885] hailo 0000:01:00.0: Probing: Added board 1e60-2864, /dev/hailo0
Example Code
Model Zoo
VIDEO WALK THRU
EXTRA COMMANDS
gst-inspect-1.0 hailotools
lspci | grep Hailo
uname -a
v4l2-ctl --list-formats-ext -d /dev/video0
ls /dev/video*
ffplay -f v4l2 /dev/video0RESOURCES
https://www.raspberrypi.com/products/ai-kit/
https://www.raspberrypi.com/documentation/accessories/ai-kit.html
Let me know in the comments if you liked what you saw, how I can improve and what should I show next? Thanks, hope to see you soon at a Meetup in Princeton, Philadelphia, New York City or on Youtube.
Get Milvused!
Read my Newsletter every week!
For more cool Unstructured Data, AI and Vector Database videos check out the Milvus vector database videos here:
https://www.linkedin.com/company/zilliz/
https://www.linkedin.com/in/timothyspann/
