Can AI solve the VAR headache?

James Courtney
Aug 14, 2018 · 5 min read

Football has taken a number of successful steps forward in recent years, with the introduction of goal-line technology and vanishing spray. So why has video assisted refereeing (VAR) been so controversial from the start?

“Ridiculous and Shambolic” Danny Rose

“Ludicrous VAR penalty… Mad” Gary Lineker

“VAR is going to absolutely ruin football” Graeme Le Saux

Tradition vs. Technology: check out this video for both sides of the argument.

These are all quotes from famous football pundits about the highly controversial video review system that has been used in this year’s FIFA World Cup. Viewers were left waiting whilst the referee stood for minutes in the centre circle and video referees miles away were hunched over screens trying to make up their minds under intense time pressure. Players and managers alike were screaming at the referee and waving their hands in the shape of a TV screen.

Manchester United legend, Gary Neville, sums up the problem perfectly:

“There are 40 camera angles and you might say there are only 10 camera angles you need to look at, but you’re asking the VAR official, with two mates alongside him, to make a decision in 10 or 15 seconds. I’m not sure they will be able to select the angles quickly enough to get the decision back to the referee before the game has been restarted.”

Surely there is a better way?

Fan experience is crucial for the ongoing success of football. VAR has caused anger, boredom and disbelief.

Artificial intelligence could be the solution.

So, in comes one of the most exciting and disruptive areas of technology: artificial intelligence (and more specifically computer vision). Computer vision is already being used to identify objects, detect skin conditions and analyse medical imagery… but how could a series of algorithms aid VAR?

Well just as AI is helping to make better decisions in businesses around the world, it can also help augment and enhance the decision making process for VAR referees.The most feasible methods of improving the speed and accuracy of VAR would be to do to prioritise the angles on display to officials, so that they only see the most relevant angles. Fewer angles = faster decisions and/or more time to make accurate decisions.

Like Gary said, only 10 of the angles might be useful, so let’s help the referees by cutting out the irrelevant 30.

Algorithms can be trained to analyse huge quantities of data and make accurate predictions.

How this could work in practise. (in Layman’s terms)

Machine learning is a subset of Artificial Intelligence that uses statistical techniques to allow computers to learn. In this case, Machine Learning can build an algorithm that will learn which angles are relevant and which angles are not. You would do this by using subject matter experts (in this case experienced referees) to go through historic footage and tag the angles that are useful and which are not. The computer would start to learn the key factors that determine whether an angle is relevant and apply these to new footage in real time.

Examples of factors that the computers could identify include whether the whole ball is viewable, how clear the footage is, whether two body parts from different players can be seen touching, etc. The beauty of machine learning is that you don’t have to tell it the factors, it will learn them itself and it will identify factors that we (as humans) might not have even realised.

So in theory, AI could be used to reduce the number of angles shown to officials by roughly 75%, speed up the game and increase decision accuracy.

Theory vs reality: what’s possible right now?

Artificial intelligence is a field of work that is improving daily and the opportunities to streamline processes is unprecedented. However, this is a particularly difficult challenge.

Areas of computer vision such as image recognition are impressive enough in themselves and only in recent years have we been able to reach with any real accuracy.

However, analysing video clips of football to determine which show the key moment is a much more complex problem that may cause some difficulties. Here at Filament, we have identified three primary issues:

  1. Training computers to understand depth and perspectives (whether objects such as a foot and a ball are touching from a 2D image) is very challenging.
  2. There is a short time window of a couple of seconds around the decision points, which means there is limited data to draw from. In general, the more high quality and relevant training data, the better the accuracy of the future predictions.
  3. Providing the results in a timely manner is essential in this use case but video can take a long time to analyse, due to the video being broken down into separate images. It would take a significant amount of computer power to process this in time.

Final thoughts:

VAR as a concept is excellent but the execution has been poor. One major element is the fan experience, both at the stadium and watching on the television at home. There are two crucial components: transparency and engagement.

I experienced a perfect example of this when at Wimbledon. A review was shown on large screens and the fans built up a crescendo of clapping and large ‘oooohs’ erupted as the ball was shown to have narrowly clipped the line. Another example is rugby, where the referees can be heard explaining their thought process and an animated screen shows the final result.

Football can learn a lot from these successful implementations. Currently the referee stands in the middle of the pitch with a finger to their ear waiting for a result. Fans in the ground don’t know what is going on and fans at home are shown pictures of three referees staring at screens. No transparency and no fan engagement.

What do you think? Let me know in the comments.

What else can AI do:

Understandably, people get very excited by the possibility of being able to replicate the human brain and imagining situations similar to that of the television show Humans. However, the area that can really make a difference to the world right now is Applied AI.

Applied AI is the use of machine learning and neural networks to solve real business problems. We at Filament are experts in Applied AI and have added significant business value to a whole range of clients from a variety of industries. Here are some examples:

Computer vision can be used to identify anti-social behaviour in crowds and make construction sites safer.
Chatbots can understand messages from customers and respond in an intelligent way.

If you want to find out more, check out our website. Thanks for reading and let me know your thoughts in the comments section!

Filament-AI

We are a team of designers and developers focused on bringing AI & Machine Learning technologies to organisations.

James Courtney

Written by

AI Strategist @ Filament. Founder of LUX Rewards and Co-Founder of Chat Taxi. "Top 10 Entrepreneurs to Watch in 2017" SETsquared.

Filament-AI

We are a team of designers and developers focused on bringing AI & Machine Learning technologies to organisations.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade