How to get Blizzard & Google Deepmind’s PySc2 working for free on Colabs
TL;DR: If you would like to get started with a FREE StarCraft II Machine Learning environment, complete with GPU hardware, you can check out the starter Google Colab notebook here:
https://colab.research.google.com/drive/1AzCKV98UaQQz2aJIeGWlExcxBrpgKsIV
Or an optimized DQN version here:
https://colab.research.google.com/drive/18asH_hmCjgUUCb-_-SPgPWkK0MYLPAmk
Ai has mastered Chess & even recently the insanely dimensionally complex game of Go.
However outside of classic board games, real time games have many areas where current machine learning (ML) techniques fail.
Examples include, when parts of the game are hidden or what is commonly known as “fog of war”, when there are many buttons or “actions” that the machine learning algorithm needs to control, when there is more than one opponent in the game & long term planning just to name a few!
Around August last year (2017) Google Deepmind partnered up with Blizzard Entertainment to release what is essentially “the next big challenge” for Ai research.
Taking on Starcraft 2.
Before we get started, you will need:
- A computer capable of using an up to date version of Google Chrome,
- A stable internet connection,
- A Colabs optimised machine learning model (we will get to how you can do this in a little bit, but if you don’t care about how we managed to pull this off, feel free to skip to the next article in part 2 “how to optimise your machine learning model for Colabs” via this link)
Why use Colabs?
Writing a machine learning agent is hard.
If you only have access to a CPU, training a machine learning model can eat up days of your computers cycles just to get a single result.
Being human, we quickly lose interest in projects we are not able to interact with on a day to day basis. So, getting a single result every couple of days or even weeks(!) is simply an impractical way of doing research.
Also, on a personal note, it is even harder when you have a Macbook Air from 2011 & lack fancy things such as tons of RAM & a modern day GPU!!!
Enter Colaboratory
Late last year Google research released their machine learning tool, Colaboratory or Colabs for short.
The first release was pretty cool. It gave you access to a Jupyter notebook style front end which connected you to a virtual machine (VM) located on Google cloud — all for free.
However, what really peaked our interest back in late January (2018) was Google updated Colabs to give you free access to a GPU!!!
free,
access,
to,
a,
GPU,
Did we mention it was free? Holy mackerel balls.
(as a sidenote: Google is giving you free access to a VM with 11 GB of RAM & NVIDIA K80 GPU!!, last time we checked, NVIDIA K80 GPUs were going for $1500 on Amazon…)
What then could be more amazing than training your machine learning model on the the cloud, completely for free, downloading your models’s learnings, and then seeing the results on YOUR computer in 12 hours as opposed to days or weeks.
How we did it.
The easy part
We immediately went to work trying to see if the headless Starcraft 2 version released last year (aka minus fancy Starcraft graphics) could download onto Colabs.
To our surprise it worked.
!wget http://blzdistsc2-a.akamaihd.net/Linux/SC2.4.0.2.zip--2018-04-10 03:11:41-- http://blzdistsc2-a.akamaihd.net/Linux/SC2.4.0.2.zip
Resolving blzdistsc2-a.akamaihd.net (blzdistsc2-a.akamaihd.net)... 72.246.56.35, 72.246.56.72
Connecting to blzdistsc2-a.akamaihd.net (blzdistsc2-a.akamaihd.net)|72.246.56.35|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3457493499 (3.2G) [application/zip]
Saving to: ‘SC2.4.0.2.zip’
SC2.4.0.2.zip 37%[======> ] 1.20G 28.2MB/s eta 84sSC2.4.0.2.zip 100%[===================>] 3.22G 30.2MB/s in 2m 10s
2018-04-10 03:13:53 (25.3 MB/s) - ‘SC2.4.0.2.zip’ saved [3457493499/3457493499]
Next up on the agenda, was unzipping the headless client,
That worked too.
!unzip -P iagreetotheeula SC2.4.0.2.zipArchive: SC2.4.0.2.zip creating: StarCraftII/ creating: StarCraftII/Replays/ creating: StarCraftII/Battle.net/ creating: StarCraftII/Battle.net/Cache/ creating: StarCraftII/Battle.net/Cache/02/ creating: StarCraftII/Battle.net/Cache/02/4e/ inflating: StarCraftII/Battle.net/Cache/02/4e/024eaf5791a26ab0ab99e5c3ef5c2b96ebbdb176dd8862bfee38a9c3242e4115.s2ma
Of course you need Deepmind’s PySc2 in order to be able to able to get your Python code talking to Starcraft 2.
So, with a simple install command… viola! PySc2 is now installed on Colaboratory.
!pip install pysc2Collecting pysc2 Downloading PySC2-1.2-py3-none-any.whl (112kB) 100% |████████████████████████████████| 112kB 2.1MB/s
The hard part
We then managed to fire up an instance of the headless client, but when PySc2 tried to connect via a technology known as websockets, we got a connection timeout error.
Well, that was a waste of time.
Not quite.
After an extreme amount of hard work, Frank He from our StarAi team managed to debug Colabs (without a debugger…) on the root cause of the error. For an excellent explanation of how he managed to do this, feel free to head on over to his medium post here.
It turned out there was a memory allocator error causing the SC2 headless client to crash on Colaboratory.
So now, we have a WORKING version of the headless client on Colabs talking to PySC2
Game on!
Training a Machine Learning Agent
In another recent success, one of our team members William Xu, managed to get a version of a machine learning algorithm known as DQN working on the “move to beacon” problem from the initial PySC2 release.
We decided to try Will’s agent on Colabs and to our never ending surprise, it started to train!!!
!PYTHONPATH=. python ~/pytorch/sc2_agents/BaseTrainer.py --map=MoveToBeacon --train=TrueEntering load game phase.
Launching next game.
Next launch phase started:
How to Train your agent.
First and most importantly, you will need a stable internet connection. The reason being that if you disconnect from Colabs whilst the model is training all your hard work is lost.
Ideally, it is best to leave your model training overnight so you can wake up the following morning to fresh machine learning results. If you are anything like us & your flatmates like to watch Netflix, perhaps try find a better internet connection where your model can train safely!
Also make sure that your computer does not time out to it’s lock screen. We have found that this seems to disconnect the internet too.
You will also need to optimise your machine learning model for Colabs, this step is easy & we have written a follow up article and in part 2 we show you how to optimize your model for Colabs
And that is it!!
Get started now >>
Below is a link to our Colabs notebook which, if you are using Google Chrome, you can load and play with in your browser right now!
Base agent
https://colab.research.google.com/drive/1AzCKV98UaQQz2aJIeGWlExcxBrpgKsIV
DQN Agent
https://drive.google.com/file/d/18asH_hmCjgUUCb-_-SPgPWkK0MYLPAmk/view?usp=sharing
We would like to thank Blizzard, Deepmind & Google for the release of these amazing tools to the machine learning community & hope that this work will inspire others to push the boundaries of machine learning using the Starcraft 2 environment.
We have made this sound easy but truth be told it was a lot of hard work. I would also like to personally thank Frank He for the amount of time he put in debugging the memory allocator issue and William Xu for his awesome guidance around the DQN algorithm.
StarAi is a small team of Developers, Machine Learning Engineers & Researchers based in Sydney, Australia