A study of CS:GO and Machine Learning | Overview on how Machine Learning can be used to tackle the issue of cheating.
CS:GO is a game that is crowded with plenty of cheaters that cause a lot of displeasure among the community. In the hunt to ban cheaters, cheat developers have always managed to stay ahead of the anti-cheat measures. But can that change? Can the AI and Machine Learning techniques be used to counter cheats and make this race a bit closer?
Last month Gabe Newell from Valve mentioned in his AMA on /r/The_Gaben subreddit that:
Simplistically we have lots of data and compute capability that looks like the kinds of areas where machine learning should work well.
This statement gives a peek on how Machine Learning is turning out to be useful even to Valve and they do plan to utilize it to improve various aspects of their operations. YouTuber and respected community member 3kliksphilip mentioned in his video CS:GO’s Future from Valve’s AMA that perhaps AI can be used to detect new kinds of cheats, to detect abusive players and griefers in the game and even to assist overwatch. Although he sounded very excited and it seemed like wishful thinking, he thought that this would probably never happen, but he was right on the money.
A recent reply on the /r/GlobalOffensive subreddit by a Valve representative from the VAC (Valve Anti-Cheat) brings in good news with some great insight that Machine Learning techniques are already being applied to CS:GO and the early results have been promising as well!
The comment explains:
Any hard-coded detection of spin-botting leads to an arms race with cheat developers — if they can find the edges of the heuristic you’re using to detect the cheat, the problem comes back. Instead, you’d want to take a machine-learning approach, training (and continuously retraining) a classifier that can detect the differences between cheaters and normal/highly-skilled players.
The process of parsing, training, and classifying player data places serious demands on hardware, which means you want a machine other than the server doing the work. And because you don’t know ahead of time who might be using this kind of cheat, you’d have to monitor matches as they take place, from all ten players’ perspectives.
There are over a million CS:GO matches played every day, so to avoid falling behind you’d need a system capable of parsing and processing every demo of every match from every player’s perspective, which currently means you’d need a datacenter capable of powering thousands of cpu cores.
The good news is that we’ve started this work. An early version of the system has already been deployed and is submitting cases to Overwatch. Since the results have been promising, we’re going to continue this work and expand the system over time.
But what does this mean? How exactly can a Machine Learning technique be any different than a standard statistical model? Many of you probably have understood the gist of it, if not, I’ll try to explain it in layman's terms.
The Science and Math
Consider a simple spin-bot cheat used by a player in the game. The aim of the player is spinning haphazardly and for a split second aims at the enemy and shoots them when they are in sight. Considering overly simplified statistics (just for the sake of understanding the problem), we can check the crosshair movement and position of the player to determine if they are using this spin-bot or not. But this movement can be mimicked by a player who is not cheating by using extremely high sensitivity or if the player’s mouse movement bugs out. Now we need to consider other things like movement, difference in time between when the player snaps him aim for a split second and gets the kill. One can build a statistical model by considering these features (and many more) to detect a genuine player messing around with sensitivity and an actual spin-bot. But these models are easy to reverse engineer. As mentioned in the comment by Valve, “any hard-coded detection of spin-botting leads to an arms race with cheat developers — if they can find the edges of the heuristic you’re using to detect the cheat, the problem comes back.” Once they know how the model is developed and what features were used they can try and circumvent those measures. The model is not going to change hence the reverse engineering is successful in evading that model.
Enter Machine Learning, if the model keeps learning about these counter measurements then the anti-cheat gets the upper hand in this arms race with the cheat developers. Machine Learning models use various algorithms that try to mimic how human brain learns things. There are several algorithms that are helpful to classify data into two categories. The two categories we are talking about here are cheating and not cheating.
The above flow diagram tries to explain how the approach is being currently tried out by Valve. Please note that this might not be accurate and is extremely generic. This flow of Machine Learning algorithms are split into three different phases here: 1. Training, 2. Testing and 3. Validation.
In the training phase CS:GO demos are parsed for various features and fed to the algorithm. Example features can be: hit accuracy, player movement, crosshair movement, fire on aim ratio, fire on visible ratio (when the enemy is visible), time to hit, etc. Redittor /u/PM_ME_SOME_STORIES attempted to implement a neural network algorithm and they have also given a couple example papers to study which kind of features can be useful to train the model. Each of these features have weights that describe how important of a factor they are in determining if a player is cheating or not. For example, hitting headshots and time to hit are important factors so they have a chance of having a higher weight factor. All these weights are calculated during the training phase. Each demo instance that is ran through the model changes the weights according to if the demo contains cheating or not. Valve has access to thousands of demos of people cheating and not, which is an extremely rich dataset to start with and this helps to train an efficient model.
Once the model is built by using sample demos, it is time to test it. Another chunk of sample demos is fed to the model. This time the feature if the demo contains cheating or not is hidden from the model and the model has to determine if there was any cheating or not. The model uses the features and weights that it has learned in the previous training stage and classifies these samples into the two categories. If the demo is determined to contain cheating then it is sent to review in the Overwatch system where the community investigators can re-affirm the inference made by the model.
The results from the model and Overwatch are then assessed during the validation stage. Here checks are made to improve the model, false positives are fed back to the training model and the model keeps learning and improving in accuracy. A continuous loop operation of these three stages helps to keep up to date with the most recent cheats and even the counter measures taken by cheat developers.
Effectiveness and Challenges
This type of machine learning model continuously keeps updating their weights and helps to counteract the changes that the cheat developers can make to evade this kind of model. The model is almost like a black box which takes in the previous results and new training/testing data. Unlike hard-coded standard statistical model, the edges of heuristics cannot be determined, even if they are determined, the possibility of those edges/features and their weights changing is very high as the model is in a constant process of learning from past results and new data. Hence the attempts of countering are rendered futile.
Valve’s comment also explains that to setup such a system a lot of resources will be utilized as the demo is to be parsed and processed from all 10 player perspectives. Millions of match-making games are played on a daily basis so to fully analyze the demos the Machine Learning algorithms would need to be deployed in data-centers apart from the server architecture. There is on-going research on performing inference on the edge. This means that the model can be built and trained in these data-centers, but the model can be deployed along with the CS:GO servers where the demos can be classified on the “edge”. As newer algorithms are implemented and older algorithms are trained, solutions to these challenges will be found eventually.
Of course the drawback is that this system is in its infant stage and can only be used for obvious and blatant cases, spin-bots for example. As these models learn more and as more data is gathered, the subtle cheats can also be tested upon. It is still a start and the applications in CS:GO are not just limited to detecting cheating. It is a very exciting to see how the advancements in computer science can help with an ever-present problem in Counter Strike. I suggest you put on your investigator hat and start helping these Machine Learning models to get more “sentient” by doing some Overwatch cases!
Questions and feedback welcome @SidiPi7.
