Accelerating YOLOv8 Object Detection Model on AIxBoard with OpenVINO™

Author: Yang Xuefeng

1. Introduction

Let’s discover the deployment and evaluation of the YOLOv8 classification model using OpenVINO™ Toolkit on AIxBoard. We will focus on accelerating the YOLOv8 object detection model.

First, download the example code repository mentioned in this article and set up the development environment for YOLOv8 with OpenVINO™ inference engine. Use the following command to clone the code repository:

git clone

openvino_notebooks/notebooks/230-yolov8-optimization at main · openvinotoolkit/openvino_notebooks

2. Exporting YOLOv8 Object Detection OpenVINO™ IR Model

YOLOv8 has five different object detection models trained on the COCO dataset, as shown in the table below:

Start by exporting the YOLOv8n.onnx model using the command:

yolo export model=yolov8n.pt format=onnx

This will generate the yolov8n.onnx model.

Next, optimize and export the OpenVINO™ IR format model with FP16 precision using the command:

mo -m yolov8n.onnx --compress_to_fp16

3. Testing the Inference Performance of YOLOv8 Object Detection Model with benchmark_app

The benchmark_app is a performance testing tool provided by the OpenVINO™ toolkit for evaluating the inference performance of AI models. It allows testing the pure AI model inference performance without pre- or post-processing on different compute devices in synchronous or asynchronous mode.

Use the command:

benchmark_app -m yolov8n.xml -d GPU 

This will provide the asynchronous inference performance of the yolov8n.xml model on the integrated GPU of the AIxBoard.

4. Writing YOLOv8 Object Detection Model Inference Program with OpenVINO™ Python API

Open yolov8n.onnx using Netron, as shown in the figure below. The input shape of the model is [1,3,640,640], and the output shape is [1,84,8400]. The “84” represents the cx, cy, h, w, and scores for 80 classes. “8400” indicates the number of output cells for the three detection heads in YOLOv8 when the image size is 640 (80x80+40x40+20x20=8400).

Here’s an example program for YOLOv8 object detection model using the OpenVINO™ Python API:

The core source code is as follows:

yolov8_od_ov_sync_infer_demo.py

The result of `yolov8_od_ov_sync_infer_demo.py` is shown in the following image:

5. Conclusion

By leveraging the integrated GPU (24 execution units) of the AIxBoard and utilizing OpenVINO™, impressive performance can be achieved with the YOLOv8 object detection model. Asynchronous processing and the use of AsyncInferQueue can further improve the utilization of the compute device and increase the throughput of AI inference programs.

Notices & Disclaimers

Intel technologies may require enabled hardware, software or service activation.

No product or component can be absolutely secure.

Your costs and results may vary.

© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.