M2M Day 350: Reverse engineering on an airplane

This post is part of Month to Master, a 12-month accelerated learning project. For October, my goal is to defeat world champion Magnus Carlsen at a game of chess.

I’m just about to get on a plane from San Francisco to New York, on which I’ll have plenty of time to make progress on my chess algorithm.

Normally, I’d want to actually do the thing and then report back in my post for the day, but, since my flight gets in late, I’ll instead share my plan for what I’m going to do beforehand…

Two days ago, I made a little chess program that can read an inputted chess game and output the numerical evaluation for each move.

Today, during the flight, I’m going to modify this program, so that, instead of outputting a numerical value, it either outputs “good move” or “bad move”.

In other words, I need to determine the threshold between good and bad, and then implement this thresholding into my program.

I will attempt to replicate the way that Chess.com implements their threshold between “good” moves and “inaccuracies”.

I’ve noticed that, in general, a move is considered to be an inaccuracy if it negatively impacts the evaluation of the game by more than 0.3 centipawns, but I’ve also noticed that this heuristic changes considerably depending on the absolute size of the evaluation (i.e. Inaccuracies are measured differently if the evaluation is +0.75 versus +15, for example).

So, I will need to go through my past Chess.com games, and try to reverse engineer how they make these calculations.

Once I do so, I can implement this thresholding into my program.

For now, as I wait for the plane, I’ve implemented the naive version of this thresholding, where inaccuracies are described by a fixed difference of 0.3.

import chess
import chess.pgn
import chess.uci
board = chess.Board()
pgn = open("data/rickymerritt1_vs_maxdeutsch_2017-10-14_analysis.pgn")
test_game = chess.pgn.read_game(pgn)
engine = chess.uci.popen_engine("stockfish")
engine.uci()
info_handler = chess.uci.InfoHandler()
engine.info_handlers.append(info_handler)
prev_eval = 0
diff = 0
for move in test_game.main_line():
print move
engine.position(board)
 engine.go(movetime=2000)
 evaluation = info_handler.info["score"][1].cp
if board.turn:
  if prev_eval - evaluation > 0.3:
   print "bad move"
  else:
   print "good move"
 
 if not board.turn:
  evaluation *= -1
  if evaluation - prev_eval > 0.3:
   print "bad move"
  else:
   print "good move"
  
 prev_eval = evaluation
 
 board.push_uci(move.uci())
 print board

Read the next post. Read the previous post.

Max Deutsch is an obsessive learner, product builder, and guinea pig for Month to Master.

If you want to follow along with Max’s year-long accelerated learning project, make sure to follow this Medium account.