Setting up a Python Environment with Unity ML-Agents and TensorFlow for macOS

How to get started training intelligent game environments using machine learning

Image: Dan Ferguson Groove Jones

The Unity Machine Learning Agents (ML-Agents) plugin uses a Python API to train agents and game environments with machine learning methods. It’s open source and still evolving.

Since the plugin was released late last year, people have been coming up with great ideas. This is a fun sample from the Unity examples:

Unity demo: Soccer agents

For those of us new to ML, it can be a little tricky to set up and get going. If you’re looking for a Windows tutorial, the Unity install docs are very good (and were recently updated.) If you have experience with Python and ML, the Unity docs may be all you need. 👋

If you need more tips, especially for macOS, read on.👇

I decided to write up a macOS version of the tutorial since I ran into some unique challenges. I’m pretty much just sharing the notes I made for myself so I wouldn’t forget the details. 😇 In each of the steps I noted all the package versions that worked for me — newer versions may or may not work, I have not tested them all.

Let’s get started.

Part 1. Set up the Python Environment with Anaconda

Anaconda will give us all the data science packages and Python. Conda will manage your packages, dependencies, and environments. We’ll go through these steps:

Download and install Anaconda
Start and Update Anaconda
Update scikit-learn Library
Install Deep Learning Libraries

1. Go here to download Anaconda with its dependencies. Choose Mac OS Python 3.6 (or most current version) and the Graphical Installer. Double click the downloaded file and follow the installation wizard.
2. After it’s installed, verify Python installation for good measure:

python -V

Expected output:

Python 3.6.3 :: Anaconda 4.2.0 (64-bit)

4. Conda is our package and environment manager. Verify the installation and confirm version:

conda -V

Expected output:

conda 4.4.6

5. You’ll need Python 3.6, Anaconda 4.2, Conda 4.4. You may need to install some packages and confirm the updates (proceed with caution here, not sure if all updates to conda will work with ML-Unity).

conda update conda
conda update anaconda

6. We’ll also need the SciPy computing libraries. Import and confirm your SciPy environment:

# scipy
import scipy
print(‘scipy: %s’ % scipy.__version__)
# numpy
import numpy
print(‘numpy: %s’ % numpy.__version__)
# matplotlib
import matplotlib
print(‘matplotlib: %s’ % matplotlib.__version__)
# pandas
import pandas
print(‘pandas: %s’ % pandas.__version__)
# statsmodels
import statsmodels
print(‘statsmodels: %s’ % statsmodels.__version__)
# scikit-learn
import sklearn
print(‘sklearn: %s’ % sklearn.__version__)

Output:

scipy: 0.19.1
numpy: 1.13.1
matplotlib: 2.1.0
pandas: 0.20.3
statsmodels: 0.8.0
sklearn: 0.19.0

7. Then we’ll need to use scikit-learn for its data mining and analysis tools. You should update to the latest version.

conda update scikit-learn

Part 2: Install the NVIDIA CUDA toolkit

1. You’ll need to download CUDA 8.0.62. Select and download CUDA Toolkit 8.0 GA2 from the site archives (not sure if newer versions will work.) Open the zip file and run the installer.
2. Create a ‘CUDA-8.0’ folder here: HD/Developer/NVIDIA/CUDA-8.0
3. Download the CUDA Deep Neural Network library (cuDNN) version cuDNN v6.0 (April 27, 2017), for CUDA 8.0 for Mac OS.
4. Copy the ‘lib’ and ‘include’ CUDNN folders into the CUDA-8.0 folder

Part 3: Install TensorFlow

TensorFlow handles numerical computation using data flow graphs. We’ll be installing TensorFlow with native PIP.

Install PIP3

1. PIP installs and manages software packages in Python. Since we’re running Python 3.3+, you’ll need PIP3 version 8.1 or higher. *This is a list of the packages that PIP will install or upgrade: REQUIRED_PACKAGES section of setup.py
2. PIP should have already been installed with Python. Check to confirm version:

pip3 -V

Expected output:

pip 9.0.1 from /Users/'username'/anaconda3/lib/python3.6/site-packages (python 3.6)

3. If you need to upgrade to 8.1+:

$ sudo easy_install — upgrade pip
$ sudo easy_install — upgrade six

Install TensorFlow with PIP3

1. Use PIP3 to install TensorFlow from the Python package:

pip3 install tensorflow

2. Verify the version of TensorFlow installed:

$ python
import tensorflow as tf

print(tf.VERSION)

Expected output:

1.4.1

3. If TensorFlow version is older than 1.4.0, you’ll need to upgrade it. This upgrades TensorFlow to the most current version (1.1.4 in this case):

pip3 install --upgrade tensorflow

*Note: As an alternate, you could also use Conda to install:

conda create -n tensorflow python 3.6.3
conda install tensorflow

4. Once the installation complete, you can test that it was successful by launching python and running a short TensorFlow test inside the python shell:

$ python
import tensorflow as tf
hello = tf.constant(‘Hello, TensorFlow!’)
sess = tf.Session()
print(sess.run(hello))

If the system outputs the following, then you are ready to begin writing TensorFlow programs:

Hello, TensorFlow!

5. *Note — Troubleshooting Binary errors: If you’re running into any errors, there are quite a few cases covered here in Common Installation Issues. You may need to download the specific version of TensorFlow then install it:

$ pip3 install — ignore-installed — upgrade /Users/'username'/tensorflow-1.4.1-cp36-cp36m-macosx_10_13_x86_64.whl

Run the TensorFlow test prompt again to confirm it worked. Thankfully we just have to do all that just once

Now let’s move on to running the environment. 😎

Part 4: The Unity Environment

Summary: You’ll install the Unity plugin, build the Unity environment for the Balance Ball demo, and train the demo project. Then you’ll save the training scene and verify training. You can follow along in the Getting started with Balance Ball Unity docs. I’ll note some small tips below that might help.

Start here with cloning the repo for Unity ML-Agents:

git clone https://github.com/Unity-Technologies/ml-agents.git

Set up the Unity Environment

1. Make sure you have Unity 2017.3+ installed. I usually download and make a copy of the repository for good measure too. Note that I’m using ML-agents version 0.3.0b in this tutorial.
2. From the Unity startup menu, select Open project. Select the unity-environment folder in: ML-Agents > ml-agents-master > unity-environment
3. When the project loads, navigate to the folder Assets/ML-Agents/Examples/3DBall/ and open the scene.

Set up TensorflowSharp in Unity

1. Download the TensorFlowSharp plugin and double click to import into Unity. It will install into the folder: Assets > ML-Agents > Plugins > Computer
2. Set the Player Settings in: Edit > Project Settings > Player
3. Select platform PC, Mac, and Linux Standalone > Other Settings panel> Configuration section
   - Set Scripting Runtime Version to Experimental (.NET 4.6 Equivalent)
   - In Scripting Defined Symbols, add the flag ENABLE_TENSORFLOW
4. In the Resolution and Presentation panel:
   - Run in Background should be checked on.
   - Display Resolution Dialog should be disabled.

Part 5: Training the Agents

By default, the untrained models will look something like this. Balls will drop and the platforms don’t know how to move unless you attempt to move them manually (not exactly efficient):

Clumsy and untrained.

1. In the Unity Hierarchy panel, expand the Ball3DAcademy GameObject and locate its child object Ball3DBrain. Set Brain Type to External. Using the external brain will train the agents according to our Python package.
2. Build and save the training environment: File > Build settings > Build
   - Ensure that scene ‘3D Ball’ is checked.
   - Name the file and save it in the python root folder:
   ml-agents/python/’your_environment_name.app’
3. In Terminal, run the training environment:

First, use PIP to install docopt (you’ll need this to run the script in the next step.)

pip3 install docopt

Now navigate to your ml-agents folder:

cd /users/'username'/'path_to_folder'/ml-agents

And then run the script learn.py from python/learn.py with your environment that you saved previously, also in the python folder <’your_environment_name.app’> and tell it to--train

python3 python/learn.py python/'your_environment_name.app' --train

See other options for learn.py here: https://github.com/Unity-Technologies/ml-agents/blob/master/python/learn.py

4. The scene will open in a new Unity window and the models will begin training. You can alter the number of steps and speed, etc. It should take only a few minutes for the balls to learn. The models are now trained to keep the ball from falling:

Trained models. Tip top shape.

5. When the training session is complete the trained model will be saved in:
ml-agents/models/ppo/<‘your_environment_name.bytes’>

Note that you could train agents using Jupyter notebook too (optional)

Embedding a trained model into Unity

So now let’s use an Internal brain to put in our trained environment into the scene.

1. In the Unity 3DBall scene Ball3DBrain object, set the Brain Type to Internal.
2. Drag the trained output file <‘your_environment_name.bytes’> into the Unity folder Examples > 3Dball > TFModels. Then select it as the Graph Model in the 3DBallBrain inspector window.
3. Play the scene. It now runs on the the trained environment and should not drop the ball.

Well trained brains.

Congrats, we made it!

Your environment it set up and you can try running the other demo scenes. Enjoy!

Additional resources

*If you want to learn more about Python and environments:

Why You Need Python Environments and How to Manage Them with Conda

*More details about ML-Agents:

Unity-Technologies/ml-agents