Generate labeling data for the object I want automatically.

Today, I’m going to introduce how to automatically generate labeling data using YOLO v8and SiamFC to choose the object you want to generate.

❗ This post does not cover all the codes in detail.
❗❗ The code is not difficult to understand, so please refer to GitHub.
❗❗❗ If you have any difficulties/questions, please leave a comment, and I will respond promptly.

Before diving into the main topic, it would be helpful to learn more about YOLO v8 through the below link :

Auto Labeling is not a dream !

✅The main idea

I decided to create this project out of a small curiosity about why we don’t always do segmentation or detection for a single object.
we always do segmentation or detection for a single class (car, truck, etc.). So the main goal is to set a target object and automatically generate its target object information.

✅cloning the code

GitHub - TCK2001/YOLOV8_SiamFC

If you don’t want to use my data, you can download the video from anywhere you want and put it into the video folder.

-video
  -img <-- If you don't have this folder, that's okay. Don't worry about it.
  -your_video.mp4

Now, for the tracking part, we will be using one of the famous single-object tracking models, SiamFC, which can handle all of the unlabelled classes.

SiamFC architecture

[ paper link ]

✅Using own video to generate the image.

-pretrained
-siamfc
-video
  -img
  -your_video.mp4
demo.py <----- this file
...
...

# Example  
# python demo.py --model pretrained --data video

parser.add_argument('--data', help='Video file folder path', required=True)
parser.add_argument('--model', help='Model file folder path', required=True)

🔥 This step will take longer than expected.

🔥 Before generating the image, you must comment the code below in siamfc/ops.py.

roi = img[pt1[1]-30:pt2[1]+30, pt1[0]-30:pt2[0]+30]
    
# Before predicting, you have to change the data type to a NumPy array.
roi = np.array(roi)

# There are many hyperparameters in there. Please refer to the official website.
# https://docs.ultralytics.com/zh/usage/cfg/#predict
results = model.predict(roi, device='0', conf=0.25, max_det=1)
if results[0].masks is not None:
    clss = results[0].boxes.cls.cpu().tolist()
    masks = results[0].masks.xy

    annotator = Annotator(roi, line_width=2)

    for idx, (mask, cls) in enumerate(zip(masks, clss)):
        det_label = names[int(cls)]
        if det_label in objects_of_interest:
            annotator.seg_bbox(mask=mask,
                            det_label=det_label)

img[pt1[1]-30:pt2[1]+30, pt1[0]-30:pt2[0]+30] = roi

✅After preparing the image, run demo_image.py.

python demo_image.py --model pretrained --data video\img

Checklist

1. You must check whether the pretrained model is in the pretrained folder, and also check the evaluation images in the folder.

2. Already uncomment below code

roi = img[pt1[1]-30:pt2[1]+30, pt1[0]-30:pt2[0]+30]
    
# Before predicting, you have to change the data type to a NumPy array.
roi = np.array(roi)

# There are many hyperparameters in there. Please refer to the official website.
# https://docs.ultralytics.com/zh/usage/cfg/#predict
results = model.predict(roi, device='0', conf=0.25, max_det=1)
if results[0].masks is not None:
    clss = results[0].boxes.cls.cpu().tolist()
    masks = results[0].masks.xy

    annotator = Annotator(roi, line_width=2)

    for idx, (mask, cls) in enumerate(zip(masks, clss)):
        det_label = names[int(cls)]
        if det_label in objects_of_interest:
            annotator.seg_bbox(mask=mask,
                            det_label=det_label)

img[pt1[1]-30:pt2[1]+30, pt1[0]-30:pt2[0]+30] = roi

🔥Result

If you want to know more about YOLOv8, click the link below!

Custom DataSet in YOLO V8 !

Thanks !