Sign Language Alphabets Detection and Recognition | Train YOLOv8 on Custom Data

Train YOLOv8 on Custom Data

Ultralytics YOLOv8

YOLOv8 is a real time object detection model developed by Ultralytics. It is the 8th version of YOLO and is an improvement over the previous versions in terms of speed, accuracy and efficiency.

The Mean Average Precision (mAP), of 53.9 has been marked with YOLOv8, the highest ever in YOLO history.

In this article, we will cover the whole process of training YOLOv8 on custom data step by step. For implementation we will use Google Colab. The steps for training YOLOv8 on American Sign Language Alphabets dataset can be summarized as follows.

1. Importing the Required Libraries
2. Installation YOLOv8
3. Collect Data
4. Train YOLOv8 on Custom Data (American Sign Language Alphabets Dataset)
5. Validate Custom Model
6. Inference with Custom Model on Images and Videos
7. Deploy Model on Roboflow

Importing the Required Libraries

In the first, we will import all the required libraries, which include os, glob.

import os
import glob
from IPython.display import Image, display
from IPython import display

Quick check, whether we have access to GPU or not, for this we will use nvidia-smi command and if you have not set Runtime as GPU, go to Runtime and select, change Runtime type, and set the Hardware accelrator as GPU.

!nvidia-smi

Checking Access to the GPU

Creating a Helper Variable

We will create a helper variable, which will help us to easily access and navigate between different folders and sub folder.

import os
HOME = os.getcwd()
print(HOME)

Install YOLOv8

YOLOv8 can be installed in two ways from source and via pip. This is the first iteration of YOLO to have an official package.

In this article we will be use pip install method for the implementation.

!pip install ultralytics

Check if ultralytics YOLOv8 is installed and setup is complete

import ultralytics
ultralytics.checks()

Importing the American Sign Language Alphabets Dataset from the Roboflow.

!mkdir {HOME}/datasets
%cd {HOME}/datasets

!pip install roboflow

from roboflow import Roboflow
rf = Roboflow(api_key="70SpXc1OmTXzM9mxngGh")
project = rf.workspace("david-lee-d0rhs").project("american-sign-language-letters")
dataset = project.version(1).download("yolov5")

Train YOLOv8 on Custom Dataset

%cd {HOME}
%cd {dataset.location}
!yolo task=detect mode=train model=yolov8l.pt data={dataset.location}/data.yaml epochs=50 imgsz=800

Confusion Matrix

Confusion Matrix is the chart that shows how our model handle different classes.

For Alphabet A, 60% of the times, it is detected correctly for 20% of the times we get bounding box but alphabet A, is incorrectly classified, and for 20% of times, when alphabet A is there but we are not detecting it.

Confusion Matrix

Training and Validation Loss

The graph of training and validation loss shows that the behavior is correct, the model is converging. From these loses, the important ones are box_loss and cls_loss.

Training and Validation Loss

Model Prediction on Validation Batch

In the image below, the model prediction results on the validation batch is shown, these images are not strictly used for the training, so it is always better to take a look and see how our model is behaving.

Model Performance on the Validation Batch

Validate Custom Model

Now we will take the model best trained weights and will use them to validate the model. Similarly as before, we will use command line interface to do that. The difference with the previous command line interface is that we will use mode as val. Validation script is using test dataset which was not used before.

Validate Custom Model

Inference with Custom Model

Now we will take the best weights and do the prediction on Custom Images and Videos. To test American Sign Language Alphabets detection and recognition custom model, I will use a video from Youtube .

Demo Video Used to Test the Model

Output Demo

Output Demo Video

Watch Complete Video Tutorial with Step by Step Explanation:

Courses & Projects

• YOLOR Pro: course link
• YOLOR Streamlit Dashboard: project link
• Mask Detection using YOLOR: project link
• Weeds Detection using YOLOR: project link
• Car Counting on Lane using YOLOR: project link

YOLO+ Subscription

• https://www.augmentedstartups.com/yolo-plus