M2M Day 363: I waited eight hours for this?!
This post is part of Month to Master, a 12-month accelerated learning project. For October, my goal is to defeat world champion Magnus Carlsen at a game of chess.
Yesterday, I predicted that I would be able to process 150 chess games every 24 hours. It’s been 24 hours since that prediction, and, in reality, I’m processing more like 350 games every 24 hours.
In fact, so far, I’ve evaluated 10,000 chess positions (as either good or bad) and have written these evaluations to my training_dataset CSV file, ready for consumption by my machine learning model.
Although I still need a lot more data, today, at around 2pm, I started training my model on what data I had, just to see what would happen.
In the beginning, the accuracy of the model, as measured on the training data, was only slightly more than 50% (i.e. a completely random evaluation, good or bad, of the chess positions).
Eight hours later, after 10,000 iterations through the 7,000 chess positions (3,000 chess positions have processed since then), the model’s accuracy on the training data leveled off at around 99%.
At first, I didn’t realize that this accuracy was based on the training data, so I thought this was unbelievably good. In other words, I thought this accuracy number represented how the model performed on data it had never seen before (while in reality the model was optimizing itself around this exact data).
When I tested the model on genuine test data (i.e. data the model truly had never seen before), it was only able to correctly evaluate the position about 70% of the time, which is not great.
I’m hoping I can improve this performance with a lot more data, but I may also need to add sophistication to the model.
Of course, the more data, the longer the model is going to take to run, which means the slower I’ll be able to iterate on my approach.
I could really use some special hardware right about now to speed things up… I may need to look into cloud compute engines to give me a little extra boost.
I’m still making progress, but a 70% performance is a bit of a letdown after I waited nearly eight hours to see the results.
Read the next post. Read the previous post.
