Statistical Inferences of a Chess Player

By Akhilesh Dongre, Jatin Madan & Sreya Vallabhaneni

ABSTRACT

American grandmaster Hans Niemann, 19, is a chess streamer. Niemann moved up quickly in the world of chess. It took him only four years after his first rated match, in which he got a provisional US Chess rating of 1122, to become a FIDE master. Then, in 2018 and 2021, respectively, Niemann received his international master and grandmaster titles. He won the U.S. Junior Championship in July 2021.

Niemann was invited to the 2022 Sinquefield Cup after his ranking rose over the year. In round three, Niemann beat GM Magnus Carlsen with the black pieces. This gave him a live rating of 2700 and a score of 2.5/3 in the competition. But Carlsen suddenly dropped out of the competition just before the next round, without giving an official reason. After Carlsen dropped out, there was a lot of talk about how Niemann’s recent results in the Sinquefield Cup and other tournaments may have been unfair. After the competition on September 11, 2022, Niemann’s live rating was 2698.

While his performance in some of these games is within the bounds of some statistical plausibility, there is very little chance that any one player could have performed so brilliantly in so many games. Theoretically, we explain how Hans Niemann’s gaming stats will be compared to those of other players from different tiers. Before we wrap up, we look at the statistical results of Hans Niemann’s cheating in live and online chess games over the past two years.

Cheating in chess is when a player or team intentionally breaks the game’s rules or engages in other activities to give them an unfair advantage. Using an outside tool to simulate movements is considered cheating. Players use apps like Stockfish, Lila, and Alpha 0 to get ready for their games. The use of the program while playing games is also permitted, which is regarded as cheating. A Grand Master-level player only has to be aware of whether the board’s present position is balanced or not to influence games in their favor.

Hans Niemann

I. TIMELINE OF EVENTS

September 4, 2022

• With the black pieces, Hans Niemann defeats Magnus Carlsen in a game of the Sinquefield Cup in St. Louis.
• An interview with Hans follows the game.

September 5, 2022

• Magnus withdraws from the event via Twitter, linking to a video clip in which a soccer manager is heard saying, “I prefer not to speak. I’m in severe trouble if I speak.”
• Chess.com sends Hans a private email informing him that his account has been quietly closed and his invitation to the CGC has been withdrawn. Chess.com’s standard approach has been to handle account suspensions/closures and invitations for titled players (such as Hans) in a non-public manner.

September 6, 2022

• Hans publicly addresses his ban by Chess.com stating that, although he cheated a few years ago when he was 12 and 16 years old, he has never cheated “in a tournament with prize money,” “when I was streaming,” or “in a real game.”

September 27, 2022

• Magnus delivers a public statement in which he explains his “unprecedented professional decision to withdraw” from the Sinquefield Cup. He “ believes” Niemann has cheated more — and more recently — than he has officially admitted.”

II. DATA

In our view, there is no direct evidence proving Hans cheated at the September 4, 2022, game with Magnus, or that he has cheated in other OTB games in the past. But we believe certain aspects of the September 4 game were suspicious, and Hans’ explanation of his win post-event added to our suspicion.

“He’s the bad boy of chess. But did he cheat?”

Consistent with the above statement, we are prepared to show within this report that he appears to have cheated against multiple opponents in several games with the help of the data of the games he had played online and OTB (Over the Board) in PGN format (Portable Game Notation), which is the standard format for recording a game in a text file that is processible by computers and is available on following official sources and online platforms.

1. FIDE: The International Chess Federation is the governing body of the sport of chess, and it regulates all international chess competitions. So, the data of Hans Niemann games and similarly rated players can be used for comparison from this platform.

2. ChessBase: ChessBase has a whole bunch of different theme keys that give you quick access to your archives of games organized by player, tournament, site, opening, endgame, tactical themes, etc. You can sort these lists with one click, and don’t need to create a separate search to find particular types of games.

3. Chess.com: This is an internet chess server, news website, and social networking website. It openly uses machine learning to predict which moves might be made by a human in any given position. It has different models of individual famous chess players, and you can play against them.

Hans Niemann ELO Rating from 2018–2022

III. ALGORITHMS

1. Stockfish: Stockfish uses the alpha-beta pruning search algorithm. Alpha-beta pruning improves minimax search by avoiding variations that will never be reached in optimal play because either player will redirect the game. Since it is often computationally infeasible to search until the end of the game, the search is terminated early when it reaches a certain depth. Search depth is measured by ply, where a ply is a turn taken by a player. A search depth of D indicates that the distance between the root node and the leaf nodes of the search tree is D ply. Stockfish incrementally increases the depth of its search tree in a process known as iterative deepening (Groot, 1978). However, when a nominal search depth of D is reported by chess engines, it does not mean that the search has considered all variations of D moves. This is due to heuristics which cause the engine to search promising variations to a greater depth than nominal and less promising variations to a lesser depth than nominal.
2. Average Centipawn Loss (aCPL): Average centipawn loss is the number of hundredths of a pawn by which a player deviated from the most accurate move calculated by a computer. The average centipawn loss (aCPL) is the amount by which a chess engine’s evaluation of the position changes after each of the player’s moves. These values play no formal role in the game but are useful to players, and essential in computer chess, for evaluating positions. The top computer move will lose zero centipawns, but lesser moves will rest deteriorate position, measured in centipawns. This value can be used as an indicator of the quality of play. The fewer centipawns, one loses per move, the stronger the play.
3. Standard Deviation Centipawn Loss (STDCPL): The standard deviation measures the amount of CPL dispersion. A low standard deviation indicates that the values tend to be close to the mean of the CPL, while a high standard deviation indicates that the values are spread out over a wider range.

IV. EVALUATION

StockFish 14.1 is used at depth 20 for the analysis of players. A linear regression line predicts the trend for each player’s ACPL and STDCPL.

Players who have had similar growth to Niemann

1. Comparing with Erigaisi Arjun

• GM Erigaisi Arjun: 19 years
• World Ranking- 26
• Elo: 1800–2600
• Range for ACPL: 60–20
• Range for STDCPL: 85–38

At 2100, Erigaisi played like a 2100-rated player and got progressively better, making fewer mistakes as his rating got higher. On the other hand, while his rating was in the 2300s, Hans played like 2700 players. His mistakes did not go lower gradually. The aCPL was in the 40s when his rating was 1800 and it decreased gradually to 25 till his rating reached 2600 which is expected. As we can see the standard deviation in CPL loss also decreases.

2. Comparing with Magnus Carlsen

• GM Carlsen Magnus: 34 years
• World Ranking- 1
• Elo 2200–2882
• Range for ACPL: 32–17
• Range for STDCPL: 55–30

Magnus is considered the greatest of all time and has been world rank 1 for the past 15 years. Magnus has the highest player rating. His aCPL has dropped around 15 points while he progressed from 2200 to 2800. These are more than 900 games of his being analyzed over the last 10+ years. On the other hand, while his rating was in the 2300s, Hans played like a 2700 player. His mistakes did not go lower gradually. The aCPL was in the 40s when his rating was 1800 and it decreased gradually to 25 till his rating reached 2600 which is expected. As we can see the standard deviation in CPL loss also decreases.

3. Comparing with Fabiano Caruana

• GM Fabiano Caruana: 30 years
• World Ranking- 8
• Elo 2200–2800
• Range for ACPL: 35–16
• Range for STDCPL: 60–32

Caruana is an American Chess prodigy who has been in the top 10 players in the world for the past 4 years. The prodigy who broke the record held by GM Hikaru Nakamura for America’s youngest chess grandmaster, Fabiano Caruana, has climbed the mountain of chess and reached the summit. His aCPL has dropped around 16 points while he progressed from 2200 to 2800. These are more than 1000 games of his being analyzed over the last 10+ years. His standard deviation ranges for CPL from 60 to 32, like Magnus, another GM.

4. Comparing with Igors Rausis

• GM Igors Rausis: 61 years
• Banned for Cheating
• Elo 2400–2600
• Range for ACPL: 26–19
• Range for STDCPL: 42–30

Igor Rausis was banned from chess for cheating, As the graphs indicate he had aCPL of around 26 when his rating was in the 2400s, which is very dissimilar to GMs. Also, a steep decrease can be seen in aCPL for Igor.

HANS NIEMANN

• GM Niemann Hans Moke: 19 years
• World Ranking- 41
• Elo 1800–2600
• Range for ACPL: 50–20
• Range for STDCPL: 95–40

While Hans Nieman showed a natural progression from ELO 1800 till ELO 2200 and his CPL variance also depicts that. He has ACPL around 35 when he reached 2300 which is a bit more when compared to other Super GMs. When we look at the CPL standard deviation, the loss remains in the range of 60 to 40 with playing 10 games with a 100% correlation to stockfish in a Chess.com tournament. The Below graphs show progress from 2300 to 2600, making us more suspicious of cheating.

As we can see, variance in CPL i.e., making errors is somewhat constant, indicating a non-human behavioral pattern in the play style. The data points indicate a very erratic pattern which infers that he sometimes plays at a different level and sometimes plays like a computer. If the error remains constant, then his rating should not have increased but if he has any external help during the game, then a solid statistical inference of cheating can be made.

V. CONCLUSION

• There is typically a negative correlation between average centipawn loss and rating. However, Niemann’s games appear to show a different trend: his average centipawn loss increased during some periods of his rating gains.
• Niemann’s aCPL is around 26 which indicates his rating should be in the 2500s which is not the case instead it’s around 2700 where his aCPL should be around 20.
• Similarly, Niemann’s STDCPL is around 48, indicating that his Elo should be around 2500.
• Niemann’s aCPL and STDCPL show a linear decrease till 2018, around his 2300 ELO. After 2018 his rating has increased but the aCPL and STDCPL indicate a player with a rating of 2500.

VI. INFERENCES

“Data are just summaries of thousands of stories — tell a few of those stories to help make the data meaningful.”

As the conclusion suggests, a non-human pattern can be observed in the analysis. We do not have any physical evidence to support the fact that Hans had external help but looking at the statistical information and analysis mentioned above. We can observe erratic behavior while playing some games.

For more on Stockfish :

Stockfish

For more on aCPL and Cheat Detection:

ACPL And Cheat Detection

Copyright 2022 Akhilesh Dongre, Jatin Madan, Sreya Vallabhaneni

Permission is hereby granted, free of charge, to any person obtaining a copy of this article and associated documentation files, to deal in the article without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the article, and to permit persons to whom the article is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the article.

THE ARTICLE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT, OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE ARTICLE OR THE USE OR OTHER DEALINGS IN THE ARTICLE.