Virtual Saber Aid
Approaching Saber through Computer Vision and Machine Learning
This model can be used by opening up a Google Colab notebook and accessing my GitHub. A video on setting up the model can be found here, on my Youtube channel
Sports tap into our primal instincts, running, throwing, hitting, winning and losing. The skills and perceptions that allowed us to survive our harsh history are displayed in our modern athletics. From an evolutionary point of view, one of our more recent developments is hitting each other with sticks and thus fencing continues to enthrall us and satisfy a deep seated urge.
The challenge becomes translating a brutal survival instinct into a repeatable and objective game. Often points are awarded through intuition relying primarily on experience and our innate ability to recognize patterns while trying to assign attributes such as commitment, aggression and attacking to the fencers. This has garnered saber with phrases such as “More art than science” at best or more commonly being dismissed as “Alchemy”.
The Basis of Saber
The fundamental attribute of saber is acceleration. As a person swings a stick they are exerting a force on it and that force is translated into an acceleration. The human mind has evolved to have an intuitive sense of acceleration. When we catch a ball our mind can understand that it will fall at an accelerating pace. When we walk we know and compensate for our acceleration towards the ground. When a referee watches a saber bout, the referee is looking at the acceleration of the fencers. The referee may use phrases such as commitment, Right of Way, attacked first, but at a base level the referee is reacting to their internal feelings on how the two fencers accelerated.
Machine Learning
If I was more arrogant I would claim that I could come up with a mathematical interpretation of Right of Way with clear cut definitions and unassailable reasoning. Even if I perfectly represented my beliefs of saber, it would understandably never be accepted.
Despite an abundance of video examples there is no collection of official “correct” saber calls. The problem is not a lack of will but a lack of consensus. The rules of saber are in constant unofficial flux. The conventions and interpretations wax and wane from year to year or even within the season. Any given tournament, even at the highest levels, is at risk of having the rules retconned into a new interpretation. I present this not as a fault, but as part of the evolution of the sport that the rules must tolerate.
Though where I will fail, Machine Learning may succeed. Machine Learning uses a set of data to infer a pattern. The obstacle is having enough data and telling it what to look for. The data is available through thousands of high level fencing matches on the internet. And the attribute that I have chosen to look for is acceleration. To be more specific, I focus on the horizontal acceleration of the bell guard towards the opponent.
I break every fencing action down, frame by frame into four components, the Left and Right bell guard accelerations and if the hit lights are illuminated. A 1 second encounter at 30 fps can be described using a list of 120 numbers, of which 60 of those are True/False.
The strongest advantage of a Machine Learning model is that it is not set in stone but is defined by the data put into it. Touches are put into the model as either Left, Right or Simultaneous. If at a later time a touch is reversed, then the model can easily be updated and rebuilt.
To further flex with changing times, there is not even a need for everyone to use the same model. The model is meant as a director’s aid and not a replacement, so at a local or regional level it can be used to simulate the calls you expect higher level directors to make. It is reasonable that some areas may find a set of touches that better reflect their regional directing compared to another. There is zero expectation that this model would be immutable or universal.
Challenges and Limitations
The biggest challenge thus far has been limited data, I only have about 250 clips, and the accuracy of the Computer Vision tracking. Poor lighting, compressed video or a populated background can throw off bell guard detections, implying an action that through observation we know is incorrect. One solution would be higher resolution video or tracking devices on the guards but I am of the mindset that anything a person can do, a machine can be taught to do.
That is why I prefer to work with any clip that I, as a person, can visually understand. I have found that working with motion detection, expected positions and human pose approximations helps supplement the detection data and provide a more reliable position estimation. This has limits but will continue to be refined with time.
A second limitation is computing power. It is not coincidental that I chose to use Google Colab to take advantage of its image processing muscle. Analyzing a clip takes about 1 minute per second of fencing clip. This can further be optimized but the model is still quite a ways from being used as a real time resource where a 3 second analysis time would be roughly the limit.
Future
The largest barrier to entry for Saber is the rules. With the advent of the internet we can now watch a nearly unlimited amount of world class saber fencing and directing. Even though the general feel and execution at the highest levels is available to everyone, the details and nuance of the rules is still shrouded in equivocation and subjectivity. That is how a Virtual Saber Box advances the sport. It provides a consistent assessment regardless of rank, seniority or authority to anyone with a video and the internet. It provides a pathway for small clubs to improve themselves and compete against top athletes.
