Aug 9 · 3 min read

Given the recent superhuman performances of AI models in Go and complex real-time strategy video games like StarCraft, it would seem the good old Chess board offers no challenge whatsoever for today’s advanced algorithms. But what if an AI could learn to play Chess based only on human game commentary? That interesting task has been tackled by SentiMATE, a new Chess algorithm proposed by University College London.

Researchers extracted Chess commentary texts describing the quality of moves, then used sentiment analysis to capture the underlying emotions of the commentators and teach the model to play Chess.

SentiMATE is an end-to-end deep learning model that employs natural language processing (NLP) to perform an evaluation function assessing Chess move quality. This pretrained sentiment evaluation function is used to optimize the AI’s decision-making process in actual gameplay.

Unlike other Chess engines, SentiMATE does not require extensive manual feature data or large amounts of specific domain knowledge. The research team used a relatively small database of 2,700 annotated Chess game commentaries available online, including information on the various moves’ strategic value along with commentators’ discussion and assessment of the moves.

Researchers used Flair, an NLP model developed by Zalando Research, to split the commentaries into two classifications: Quality and Non-Quality. Flair combines long short-term memory (LSTM) and appropriate word embeddings to effectively learn word usage regarding Chess games. With a structured dataset containing both text-based commentary and state representation (before and after the move has occurred), researchers formatted the state representations into the evaluation function, as shown in Figure 1 below.

Figure 1. Complete pipeline for training the evaluation model

The achievements of SentiMATE are promising. The model beat both a random strategy agent and an implementation of DeepChess, the first end-to-end machine learning-based model for Chess. In 100 games playing as both black and white, SentiMATE achieved an 81 percent win rate against the random agent and defeated DeepChess on search depth 1 (indicating the number of moves ahead the agent will explore).

Figure 2. Material Score decay over time averaged over 100 games for Random and Black and White for DeepChess

SentiMATE’s novel Sentiment Analysis model based on Chess commentary introduces a new method for NLP-based move evaluation. Researchers believe the method has the potential for practical applications beyond Chess, for example sentiment analysis could also enable machines to use emotional content in tasks such as financial activity prediction, sports commentary, and recommendations.

The paper SentiMATE: Learning to play Chess through Natural Language Processing is on arXiv.

Author: Hongxi Li | Editor: Michael Sarazen

We know you don’t want to miss any stories. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.

Need a comprehensive review of the past, present and future of modern AI research development? Trends of AI Technology Development Report is out!

2018 Fortune Global 500 Public Company AI Adaptivity Report is out!
Purchase a Kindle-formatted report on Amazon.
Apply for Insight Partner Program to get a complimentary full PDF report.


We produce professional, authoritative, and thought-provoking content relating to artificial intelligence, machine intelligence, emerging technologies and industrial insights.


Written by


AI Technology & Industry Review — | Newsletter: | Become Synced Insight Partner: | Twitter: @Synced_Global


We produce professional, authoritative, and thought-provoking content relating to artificial intelligence, machine intelligence, emerging technologies and industrial insights.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade