Deep Learning for Detecting Football Players Using Convolutional Neural Network, Tensorflow and OpenCV
Case Study : Belgium National Team vs Japan National Team at the 2018 World Cup
In this article I will discuss about the use of deep learning for sports, especially in soccer. Deep learning is a branch of Artificial Intelligence (AI) and the system in Deep learning itself is inspired by the concept of Artificial Neural Networks (ANN)and usually consists of a large number of Neural Networks (NN).
The method I use is CNN (Convolutional Neural Network), and uses libraries in python, namely tensorflow and opencv. Tensorflow’s own function is to help conduct experiments on deep learning models, while opencv functions for real time computer vision programming, from opencv it can also help us in detecting a color that I use in detecting jersey colors of soccer players. The process of detecting colors from opencv, you can see in more detail here.
For more detailed explanation about Artificial Intelligence, Deep Learning, CNN and others I will discuss in the next article. Next I will discuss the stages of the object detection process for soccer players using CNN, tensorflow and opencv.
Label to Soccer Players
In the first stage I label each player with the label name “person”. It’s okay if you want to give a different name, maybe the most important thing is that we give just one name label to the object. Because on the object that we label is only one, namely soccer players. The picture I got to give a label is a frame piece from a video of a football match between the Belgian national team against the Japanese national team from the official FIFA youtube channel. As well as the number of images that I labeled 500 images.
You can download here the platform to label football players.
Convert XML to CSV and Generate TFRecord
After labeling the object, the output that has been labeled has an xml extension. Then the file will convert from xml to csv. Then the next step is the generate TFRecords which functions as input data to the TensorFlow training model, to see the code you can see here.
File convert xml to csv is xml_to_csv.py and generate TFRecord is generate_tfrecord.py
Create Label Map and Configure Training
The label map tells the trainer what each object is by defining the class name mapping to the class ID number, for example as below. Then we save the text with labelmap.pbtxt .
item {
id: 1
name: 'person'
}Next step, training object detection pipelines must be configured. This defines which model and what parameters will be used for training. This is the last step before running the training. In this process I used training model Faster RCNN Inception V2 coco, because to get high accuracy it only takes a few hours of training process. You can download it here.
Training Process and Test Results
After we have completed some of the previous steps, the next step is the training process. In this training process I used 26489 steps in the training configuration. The number of steps takes a very long time which is 7 days more, depending on the specifications of the device used. This research I used the following laptop specifications.
- Windows : Windows 10
- Processor : Intel® Core™ i5–7200U
- CPU : 2.50GHz (4CPUs), ~2.7GHz;
- RAM : 16GB DDR3
After the training process, we will do the test results. The code that I use can you see here and I use jupyter notebook to do the test result. Here are some screenshots of the results of testing.
Benefits of This Research
The implementation of deep learning is very beneficial for our lives, one example of which is soccer. In this study, the use of deep learning using the CNN method made it possible to help football observers and FIFA in analyzing a match. I made this research because it was also inspired by the following video below, so that the results of this study can help total color blind sufferers (Monochromation) to see football matches by utilizing the OpenCV Library in categorizing players’ names and national team names.
Maybe that's all I can share with you, hopefully it's useful. The key to success in conducting this research is by being patient and disciplined. If you want to learn about detection objects easily, you can visit the following this site or in Indonesian you can visit this site.
If you have questions, suggestions or criticisms you can use this platform or can send to my email galangimanta557@gmail.com.
Thank You..!!
