Installing SpinningUp for Mac OS X
Brian S. Haney
This tutorial explains how to get started with OpenAI’s Open Source platform SpinningUp. SpinningUp is an open source Python Library for training deep reinforcement learning algorithms. Indeed, OpenAI open sources six variations of deep reinforcement learning algorithms testable in its Gym environments. The six algorithms are: Soft-Actor Critic (SAC), Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), Twin Delayed DDPG, Trust Region Policy Optimization, and Vanilla Policy Gradient. Gym is a set of virtual environments within the OpenAI Open Source Library — and available through SpinningUp through command line API’s.
Step 1
The first step is to download and install Anaconda. Anaconda is a tool developers use to create virtual environments. The advantage of creating virtual environments is developers can run their research experiments in a clean environment on their machine — in our case the Mac. Once you click the link, navigate to the Anaconda Documentation page and click the macOS installer button. And, download Conda’s newest Python version.
Step 2
The second step is navigating to your terminal. And, open a new window.
Step 3
The third step is to create a virtual environment on your machine. To do so, run the following code from the terminal:
conda create –n spinningup python=3.7
Step 4
The fourth step is activate the virtual environment. Activate the virtual environment by running the following code from the terminal:
source activate spinningup
You are now in a virtual environment.
Step 5
The fifth step is to install system packages with Homebrew. Homebrew is an open source package management software, simplifying the installation of packages on MacOS. From the Terminal run:
brew install openmpi
Step 6
The sixth step is to clone and install SpinningUp. From the Terminal run:
git clone https://github.com/openai/spinningup.git
cd spinningup
pip install –e .
Step 7
The final step is to check the installation. From the Terminal run:
The output will be a series of epochs looking like this:
The default setting is 200 epochs for training. After the training is complete, you can watch a video of your trained agent by copying and pasting to the terminal:
You can post your agent’s performance on the OpenAI Leaderboard. And, you can find my solution to the LunarLander and LunarLanderContinuous environments on my GitHub. The corresponding videos are posted on my YouTube Chanel. With that, you will have installed OpenAI’s SpinningUp. And, you are now ready to begin conducting deep reinforcement learning research.
