Setting up Retro Gym in the Cloud

I have been playing around with OpenAI’s Retro, which allows you to use gym with old Sega Genesis and Nintendo games. I’ve been doing some experiments on the original Sonic the Hedgehog on my home computer and some cloud instances. I used Google Cloud and Floydhub.com for a bit before choosing Paperspace. Paperspace offers GPUs for $.40 an hour (plus $6 a month for storage). The Paperspace offering I’ve been using is a Machine Learning in a box Ubuntu 16.04 instance that lets you either terminal in or remotely use the desktop. Having desktop access is useful for rendering Sonic runs.

Here’s how I’ve been setting up my instance in a terminal

Set up an environment (optional):

conda create --name gym36 python=3.6 

Install OpenAI baselines for access to various Reinforcement Learning (RL) algorithms:

git clone https://github.com/openai/baselines.git
cd baselines
pip install -e .

To see some implementations of OpenAI baselines with RL environments see the retro baselines repo or my previous posts implementing baselines with the Pygame Learning Environment.

Install gym-retro:

git clone --recursive https://github.com/openai/retro.git gym-retro
cd gym-retro
pip3 install -e .

Install retro contest repo (optional) and retro baselines:

git clone --recursive   https://github.com/openai/retro-contest.git
pip install -e   "retro-contest/support[docker,rest]" 
git clone --recursive   https://github.com/openai/retro-baselines.git

For more info on the recently concluded Sonic contest and a link to the Discord that can help with installation troubleshooting see the OpenAI contest page.

You will also need to acquire the rom for the game and install it in the gym-retro/data/game_name/ directory.