M2M Day 362: If I didn’t figure this, there would be no path forward. Luckily, I did.
This post is part of Month to Master, a 12-month accelerated learning project. For October, my goal is to defeat world champion Magnus Carlsen at a game of chess.
Yesterday, I finished building my chess algorithm pipeline, which can take chess games downloaded from the internet, convert them into a machine learning-friendly format, and then use them to train a machine learning model. At the end of the pipeline, the program outputs a list of algorithmic parameters that I can memorize and eventually use to compute chess evaluations in my brain.
This was great progress, but, as of yesterday, there was still a problem: The part of my program that is meant to label each chess position with either “good” or “bad” (so that the machine learning model knows how to properly evaluation the position) didn’t actually work at all…
In other words, the program outputted “good” and “bad” labels nearly at random, not actually tied to the true evaluation of the chess position.
Clearly, if my data is poorly labeled, even if I can build an incredibly accurate machine learning model, the resulting chess algorithm will still be just as ineffective as the data is bad.
So, today, I worked on addressing this problem.
In particular, I continuously tweaked my program until its outputs matched the “true” evaluations of the positions.
As my source of truth, I used the website lichess.org, which allows users to import and analyze chess games, explicitly identifying all the bad moves (inaccuracies, mistakes, and blunders) in the given game:
After about 30 minutes of work, I was able to get my program to exactly mirror the evaluations of lichess across every chess game that I tested.
So, I was finally ready to generate my correctly-label dataset.
As of yesterday, the program in my pipeline that converts chess games into CSV files could only handle one game at a time. Thus, today, I augmented the program so that it can cycle through millions of games on its own.
I’m using the chess database from lichess.org (which contains over 200 million chess games) to generate my properly label dataset.
Right now, I’m running all of the September 2017 games (which is about 12 million games) through my program.
Because the program needs to deeply evaluate each position, using the Stockfish chess engine with a depth of 22, the program is taking a long time to run. It looks like it will only be able to crunch through about 150 games every 24 hours, or approximately 6,000 chess positions, if I continue to run it just on my local computer.
Also, I can only use half of these chess positions for my training dataset, given that I’m currently only training my model to play with the White pieces.
I’m not sure how many chess positions I’ll need in order to train a decently functional model, but I’m guessing somewhere between 25,000–50,000 should be enough, which will take about a week to generate at the high end.
With that said, the program is iteratively writing each evaluated chess position to the CSV file in real-time, so I can start playing around with the data that’s already processed as the other games are still processing.
Read the next post. Read the previous post.
