IoT Replaces On-court Line Umpires in Australia Open Tennis

Mustafa Abusalah
Analytics Vidhya
Published in
3 min readFeb 14, 2021

In Australia Open Tennis 2021, no more need for on-court judges anymore, Sensors and AI does the job on real time.

In the past, players had the chance to challenge the line judges decisions for a limited number of times during a game, Cameras monitoring the lines will replay and show if the ball was in or out. But in many cases, judges make wrong decisions and players run out of challenge attempts.

Research and innovations started from early 1970’s to electronic line judge, but now with the current advances in sensors, cameras and Artificial Intelligence (AI) algorithms, it became reality.

Dan Evans vs Karen Khachanov Dramatic match with horrible on-court lines judgements

Due to Covid-19 and to utilize the improvements in Internet of Things (IoT) and sensors, Australia Open Tennis decided to replace the on-court line judges with a Live electronic line calling that is delivered through remote tracking cameras covering the court. The cameras are connected to a smart AI system which automatically sends recorded audio line calls in real time.

“If a player is requesting that they would like to see a point-ending shot that has not animated automatically, then play can be stopped if the chair umpire is of the opinion that the request is reasonable, and the chair umpire should announce the replay. Play should not continue until the shot is shown on the big screen.” Australian Open Tournament Director Craig Tiley said.

Line Camera is is shown in the red box. Image source AO Facebook page.

Ball tracking technology

The ball tracking technology is not new, it has been used in Tennis when the player disputes line umpires decision and also used in Football (Succor) for goal or no goal decision. In ball tracking high quality cameras are used to track the ball.

Hawk-Eye is a computer vision system used in numerous sports such as Tennis and Football (Succor), to visually track the ball course and display a profile of its statistically most likely path as a moving image. The onscreen representation of the track results is called Shot Spot.

A study made by Yulin University, suggests that 33% of Tennis player hawk-eye challenge were successful. There is no denying the fact that the human visual judgment is limited, due to many facts including: the weather, Angle, obstructions, speed, distance, height, light and shadow, parallax, depth and color. The influence of such factors as these will directly affect the accuracy of the line umpire judgements.

Hawk-Live

Hawk-Eye ball calling system have been used for the last 14 years for line judgement challenges, the new Hawk-Live has few additions.

The Hawk-Live system is based on triangulation principles that make use of visual images and timing data provided by 12 tracking and 6 foot-fault high-speed video cameras placed at strategically locations and angles around the Tennis Court. The system uses predefined embedded model that simulates the court and includes data on Tennis rules which allows rapidly processing the video feeds through the means of a high-speed camera and a ball tracker system.

In each frame sent from a camera, the system identifies the group of pixels which corresponds to the image of the ball. It then calculates for each frame the 3D position of the ball by comparing its position on at least two of the physically separate cameras at the same instant in time. A succession of frames builds up a record of the route along which the ball has traveled.

The system then generates a graphic image of the ball route and the Tennis court lines, allowing information to be provided to the umpire model system, which plays the role of line judges.

Hawk-Live uses public address system (PA system), which is an electronic system comprising microphones, amplifiers, loudspeakers, and related equipment. The PA system is used to run recorded voice of Judges for real time ball or foot-fault calls.

In addition the trained AI system, a group of review official set inside a room to monitor the system and in case of fault they interfere.

It is expected that the model of Australia Open line umpire system will be used in the future Tennis ATP and WTA tours. The system is still not ready for Clay courts but was tested successfully on hard and grass courts.

--

--

Mustafa Abusalah
Analytics Vidhya

Manager of Learning & Innovation. Expert in Information Retrieval, Machine Learning, NLP & KM.