A New World of Performance Insight from Broadcast Tracking Technology
Computer Vision & AI in Sports
Computer vision and artificial intelligence are already changing our world. There are many definitions of Artificial intelligence (AI) but most people would agree it refers to the ability of machines to interpret data and act intelligently. In effect, this means they can be trained to make decisions and carry out tasks based on the data available. Applications in different industries are increasingly evident:
- In consumer products such as using facial-recognition algorithms to unlock iPhones.
- In medical fields by examining x-rays and MRI scans to predict types of cancer.
- In the automotive industry where self-driving cars rely heavily on computer vision to make sense of their surroundings.
In the sports industry, computer vision can now be applied directly to broadcast video in order to quantify the visibility and impact of advertising as well as monitoring and identifying the precise location and movement of athletes.
Broadcast Tracking Systems for Football
Optical based player tracking systems in Football were first used in the late 1990’s and have evolved in line with digital technology and developments in camera and image processing.
Permanent cameras located in specific fixed positions around the pitch remains the primary technology solution for football leagues wishing to collect player and ball tracking data. These semi-automated systems also require a small team of human operators to resolve errors before final data delivery which adds to the overall cost base.
“This data is normally not available outside the league in question”
The data created is delivered to the key league stakeholders, however for rights and confidentiality reasons, this data is normally not available outside the league in question. Added to this restriction, different tracking systems process and classify the data in different ways so the analytics generated from different leagues are not standardised and cannot be compared directly.
Automated broadcast tracking represents a huge leap forward as it enables player, match official and ball tracking to be accurately collected from a standard television broadcast without the need for any capital investment in stadiums or human operator costs. Player position is obtained from the main broadcast camera using a dynamic pitch calibration which is applied to the main broadcast camera as it pans left and right across the pitch.
The automated nature of the technology opens up the possibility of providing accurate and affordable tracking data, at scale across many leagues. Furthermore, standardised analytics can be applied across leagues using consistent data processing and metrics to allow meaningful comparison and benchmarking of players and teams.
Tracking data is generated whenever the main camera is used, however the data generation is not continuous so no positional data can be provided when the broadcast director switches to a player close up or shows replays.
In reality most replays are shown when the ball is out-of-play.
From a technology perspective, collecting tracking data from broadcast video in Football is the combination of multiple technical components working simultaneously including:
- View segmentation: is the current frame captured by the main camera or is it from a replay or a close-up view camera?
- Detection of players, referees and the ball: detecting and classifying on one single frame.
- Homography estimation: estimating the parameter of the camera to be able to project the detection of the object from the image to its x, y coordinates on the field.
- Tracking the players, referees and the ball: adding temporal information to track the detection from one frame to the next sequential frame.
- Recognition of the players: positively identifying the individual players in question.
SkillCorner AI Powered Video Tracking Technology
SkillCorner is a Paris based start-up specialising in artificial intelligence (AI) tracking. The company has built an AI-powered video tracking technology specifically for Football based on deep learning. The system is capable of recognising, positioning and following (in real time), the players, referees and ball from any standard broadcast.
The SkillCorner tracking technology has been developed following more than 4 years of extensive research and development and the business is committed to continued investment in the platform to ensure the technology remains the market leading Football solution.
The challenges of broadcast tracking technology
The biggest challenge to overcome with broadcast tracking is moving from an initial prototype to a full production system that is capable of working with any broadcast feed, from any stadium in the world, to the required level of accuracy.
Stadium architecture and camera location can vary significantly within a given football league (ranging from Camp Nou to smaller stadiums in the Spanish Segunda), plus the unpredictable nature of meteorological conditions in outdoor stadia (in the form of snow/hail, smoke, contrasting sun and shadow etc). Solving these operational challenges has been a key driver for SkillCorner in preparing the technology for use at industrial scale.
Below is a summary of the most challenging problems that the SkillCorner technology has solved during the period of research and development to date:
- Homography estimation: in a wide range of stadium sizes without any prior information on the position of the camera.
- Tracking of the ball in 3 dimensions: at any given time, one should be able to locate the x, y & z coordinates of the ball. The broadcast feed only includes one camera at a given time, so this estimation needs to be done from a single point of view, triangulation cannot be used.
- Unsupervised player recognition: SkillCorner’s player recognition is not trained on the match prior to the match, the algorithm is able to recognise a player it has never seen before. This is essential to recognise players on their first professional match, or when they use a new jersey and to ensure scalability of the product.
- Real-time delivery: SkillCorner processes video at 10 frames per second, in real time, with a delay of <2 seconds on the tv feed — meaning the data is delivered to the client 2 seconds after the video is received.
Different uses of the technology
The technology has applications across media and betting markets as well as the performance market with clubs, federations and leagues.
- From a fan engagement perspective, the data provided by SkillCorner is used to animate a live widget showing the movement of the players on the pitch, in real-time.
- From a performance perspective, the data provided by SkillCorner is highly valuable for player recruitment where data is required across many leagues. Player tracking data allows deeper technical and tactical insights beyond event level data, plus the monitoring of physical performance at player, team and league levels.
Having built operational scalability, the SkillCorner AI platform provided automated data collection for 23 Football leagues during the 2019/20 season and is expected to increase to 40 competitions in the coming months.
