Image for post
Image for post

How to build your own AlphaZero AI using Python and Keras

Teach a machine to learn Connect4 strategy through self-play and deep learning

David Foster
Jan 26, 2018 · 11 min read

Update! (2nd December 2019)

AlphaGo → AlphaGo Zero → AlphaZero

Image for post
Image for post
Match 3 of AlphaGo vs Lee Sedol
Image for post
Image for post
A graph from ‘Mastering the Game of Go without Human Knowledge’

1. AlphaZero requires zero human expertise as input

2. The algorithm is ridiculously elegant

How to build your own AlphaZero

The code

Connect4

Image for post
Image for post
Connect4

game.py

Image for post
Image for post
Action squares for Connect4

run.ipynb

agent.py

model.py

Image for post
Image for post
A sample of the residual convolutional network build using Keras
current_player.model.viewLayers()
Image for post
Image for post
Convolutional filters from the neural network

MCTS.py

config.py

Image for post
Image for post
Image for post
Image for post

funcs.py

from game import Game
from funcs import playMatchesBetweenVersions
import loggers as lg
env = Game()
playMatchesBetweenVersions(
env
, 1 # the run version number where the computer player is located
, -1 # the version number of the first player (-1 for human)
, 12 # the version number of the second player (-1 for human)
, 10 # how many games to play
, lg.logger_tourney # where to log the game to
, 0 # which player to go first - 0 for random
)

initialise.py

memory.py

loss.py

settings.py

loggers.py

Image for post
Image for post
Output from the logger.mcts file
Image for post
Image for post
Output from the logger.tourney file

Results

Image for post
Image for post
Loss against mini-batch iteration number
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Learning a different game

Summary

Image for post
Image for post

Applied Data Science

Cutting edge data science, machine learning and AI projects

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store