Identifying posture of pigs by applying Yolov8 for instance segmentation

For better pig welfare

INTRO

In this article I will explain how to use Yolov8 for instance segmentation on a custom dataset of pigs, in order to identify and track their different postures. Why pigs you may ask. Well, instance segmentation can be use for almost anything, however, we (at UCL, www.ucl.dk) have been very fortunate to be granted access to an ultra-modern full line plant pig producer, namely Kokkenborg Aps https://kokkenborg.com/en/ in Denmark, who let us install cameras in the stables. Kokkenborg was also kind enough to let us share the images from the pigsties with the public. So if your are interested in using these images, they can be found at this link: PigsCam1ImagesAndJson. We only ask, that in any work you use the images, that you give credit to Kokkenborg Aps.

A WORD ABOUT DATA

In order to capture the images we (many thanks go out to my colleagues Mathias Elbæk Gregersenv, Jalal Azadi &Lars Kristian Lindegaard Mikkelsen, this couldn’t have happened without their help). build a box which included camera, a raspberry pi, a harddrive, and a GSM device, and then mounted the box to the ceiling (see image below).

The box we mounted to the ceiling

A new picture was taken every 10 minutes for at week from 8 am to 10 pm every day (during the light hours). Images that were too dark or which included humans have been removed from the dataset (in order to protect the privacy of the workers in the pigsty). This has resulted in 3357 images. We have annotated the pigs in the images into the following four posture categories: “standing”, “inactive”, “eating”, and “drinking”.

The annotations are done using an open source program called LabelMe https://github.com/wkentaro/labelme which outputs a json file that will fit directly on a Detectron2 model https://github.com/facebookresearch/detectron2. However, data formats vary between segmentation architectures. So whereas Detectron2 uses json files, Yolov8 uses txt files. There are other small differences as well, but you can convert between formats if necessary. If you want to follow along with this tutorial you have to Install labelme2yolo in order to get the Yolov8 format. The link to the conversion tool can be found here: https://pypi.org/project/labelme2yolo/ and the process on how to do this is shown below. Everything shown in this tutorial can also be applied to your own custom dataset as long as your data is annotated into either the format of Yolov8 or the format of Detectron2 (in which case you have to follow the conversion steps below).

SETTING UP THE PROJECT

Now I wil explain how to set up the project. First of all you have to download the data from the link provided here: PigsCam1ImagesAndJson or get your own annotated data.

Next, open your terminal and create a project directory where you want to place the data and the scripts we are going to write. Please note that this tutorial presupposes you are working on a linux platform.

mkdir Pigs_Yolov8

Go to the directory you created

cd Pigs_Yolov8

From now on we will refer to this as the project root and all terminal calls will be made from here.

Add the folder with images and json files to the project root folder and call this folder: PigsCam1ImagesAndJson.

In my case I have images and json files in two separate folders, and I simply merged them together into a new folder called PigsCam1ImagesAndJson. This is the only folder of the three you will need.

Next create a virtual environment, where we can install all the dependencies we need for the project.

Here is the command to create the virtual environment. The last word is the name of the environment.

python3 -m venv PigYolov8Venv

The virtual environment will be created inside the directory you are currently in.

Next activate the environment:

source PigYolov8Venv/bin/activate

In the terminal it will look like this:

Now you should see your virtual environment within the root directory of your project. In case you are new to virtual environments, the purpose of virtual environments is to separate installed packages in the current project from other projects on your machine so you can avoid version mismatch between projects.

Next, install the following packages into your virtual environment. Be sure to be in the root of your project folder and with the virtual environment activated.

pip install numpy==1.21.2 scipy==1.7.0 scikit-learn==1.0 matplotlib==3.4.3 pandas==1.3.2
pip install labelme2yolo
pip install ultralytics

Now we are going to convert our dataset format into Yolov8’s format

Run the following command to create a yolov8dataset (changing the directory to point to your own)

labelme2yolo - json_dir /home/simonadmin/Documents/Pigs_Yolov8/PigsCam1ImagesAndJson - val_size 0.15 - test_size 0.15

The parameters –val_size and –test_size specify how much of the dataset to use for validation and test.

…

You should now have a folder called YoloDataset which contains an images folder, a labels folder and a dataset.yaml file.

Both images and labels are divided into train, test and val folders (see below and check that it matches your own folder structure).

TRAINING

In order to train the model you will need to make sure you have torch installed and preferably running on a gpu.

You can check if torch is installed and up and running by the following commands:

In case torch is not installed go to https://pytorch.org/ and follow the installation procedure.

Now we will finetune an existing model to be able to learn our custom classes.

The model used for training can be found on the following link: https://github.com/ultralytics/ultralytics#models (go down to segmentation) which shows the following table. From this table we just need the name and the size as parameters to the training algorithm.

The actual filename is the name in blue followed by the extention .pt. so for instance. If we want to use the YOLOv8m-seg model, we need the name YOLOv8m-seg.pt

Other parameters we need are:

• The size parameter, which is 640.
• We also need to define number of epochs (how long we sill train for), in our case we set it to 100 epochs. Epochs is mostly a matter of how you you are willing to wait, the more epochs the better the result, however, there comes a point where no improvement is achieved, so starting with 100 epochs seems like a good idea.
• The batch size. This depends on your gpu, I use 8 as my batch size.
• What type of task, which is segmentation in this case.
• Which mode, namely training.
• And the path to the dataset.yaml file, which points to our data, labels and number of classes to identify. In our case this is four classes (see image below). Make sure the paths match what you expect on your machine.

Here is the command in full which you have to run now:

yolo task=segment mode=train epochs=100 data=/home/simonadmin/Documents/Pigs_Yolov8/PigsCam1ImagesAndJson/YOLODataset/dataset.yaml model=yolov8m-seg.pt imgsz=640 batch=8

And the result should look something like this

Once training is done, you will notice that some files and folders have been created in the root of your project folder.

The two .pt files are models which the script has downloaded to do its work. And the folder “runs” includes the different training runs we have performed.

Inside the runs folder, you can see graphically how well the training went or whether you should train for longer.

It looks like our training went well. Especially predicting standing is very

And here are a some examples of how well validation went.

INFERENCE

Now comes the real fun part. We are going to take our new model for a spin.

In order to test the model, you need an image and/or a video of pigs preferably taken from the same position as the training data.

Next, inside the runs folder, go into the weights folder (se path hierarchy below)

Copy the file best.pt and place it in the root folder of your project (where the other .pt files are located).

Add the video and/or image of pigs to the root as well.

CREATE A TEST SCRIPT

Create a small script called predict.py in your favorite editor, and add the following:

from ultralytics import YOLO
model = YOLO("/home/simonadmin/Documents/Pigs_Yolov8/best.pt")
model.predict(source= "/home/simonadmin/Documents/Pigs_Yolov8/PigMovieNew.mov", show=True, save=True, conf=0.5, save_txt=False, save_crop=False, show_labels=True)

Movie format does not have to .mov but can also be .mp4 or .avi

If you want to test on an image instead, then simply change the source parameter to point to an image on your machine

Place the script in the root of your project and run it with the following command

python predict.py

And here is our result:

Pretty good! The model seems to have identified the pigs correctly.

And here is the result of our video (Please note: The video has been created from multiple images which is why it looks as if it is lagging).

The video result could be better, but this would probably require a bigger dataset especially a dataset where postures are more evenly represented.

Where to go from here?

The dataset to replicate our work is available at this repository: PigsCam1ImagesAndJson and is available for use under the most open license possible, we only ask that if you use the code or the dataset, that you credit Kokkenborg.

When using Yolov8 it is possible to capture the amount of pigs in a certain category (e.g. pigs that are eating), so you can keep count of the pigs at any given moment. It is also possible to track the pigs which gives the option of creating diagrams that show for how long a specific pig has been active or inactive.

In order to draw on the video stream in real time for instance in order to track or count the pigs, follow this tutorial video made by RoboFlow about the SuperVision library:

I hope you enjoyed the article. Let me know if something isn’t working out for you.