Setting up a Python Environment with Unity ML-Agents and TensorFlow for macOS
How to get started training intelligent game environments using machine learning
The Unity Machine Learning Agents (ML-Agents) plugin uses a Python API to train agents and game environments with machine learning methods. It’s open source and still evolving.
Since the plugin was released late last year, people have been coming up with great ideas. This is a fun sample from the Unity examples:
For those of us new to ML, it can be a little tricky to set up and get going. If you’re looking for a Windows tutorial, the Unity install docs are very good (and were recently updated.) If you have experience with Python and ML, the Unity docs may be all you need. 👋
If you need more tips, especially for macOS, read on.👇
I decided to write up a macOS version of the tutorial since I ran into some unique challenges. I’m pretty much just sharing the notes I made for myself so I wouldn’t forget the details. 😇 In each of the steps I noted all the package versions that worked for me — newer versions may or may not work, I have not tested them all.
Let’s get started.
Part 1. Set up the Python Environment with Anaconda
Anaconda will give us all the data science packages and Python. Conda will manage your packages, dependencies, and environments. We’ll go through these steps:
Download and install Anaconda
Start and Update Anaconda
Update scikit-learn Library
Install Deep Learning Libraries
- Go here to download Anaconda with its dependencies. Choose Mac OS Python 3.6 (or most current version) and the Graphical Installer. Double click the downloaded file and follow the installation wizard.
- After it’s installed, verify Python installation for good measure:
python -V
Expected output:
Python 3.6.3 :: Anaconda 4.2.0 (64-bit)
4. Conda is our package and environment manager. Verify the installation and confirm version:
conda -V
Expected output:
conda 4.4.6
5. You’ll need Python 3.6, Anaconda 4.2, Conda 4.4. You may need to install some packages and confirm the updates (proceed with caution here, not sure if all updates to conda will work with ML-Unity).
conda update condaconda update anaconda
6. We’ll also need the SciPy computing libraries. Import and confirm your SciPy environment:
# scipyimport scipyprint(‘scipy: %s’ % scipy.__version__)# numpyimport numpyprint(‘numpy: %s’ % numpy.__version__)# matplotlibimport matplotlibprint(‘matplotlib: %s’ % matplotlib.__version__)# pandasimport pandasprint(‘pandas: %s’ % pandas.__version__)# statsmodelsimport statsmodelsprint(‘statsmodels: %s’ % statsmodels.__version__)# scikit-learnimport sklearnprint(‘sklearn: %s’ % sklearn.__version__)
Output:
scipy: 0.19.1numpy: 1.13.1matplotlib: 2.1.0pandas: 0.20.3statsmodels: 0.8.0sklearn: 0.19.0
7. Then we’ll need to use scikit-learn for its data mining and analysis tools. You should update to the latest version.
conda update scikit-learn
Part 2: Install the NVIDIA CUDA toolkit
- You’ll need to download CUDA 8.0.62. Select and download CUDA Toolkit 8.0 GA2 from the site archives (not sure if newer versions will work.) Open the zip file and run the installer.
- Create a ‘CUDA-8.0’ folder here: HD/Developer/NVIDIA/CUDA-8.0
- Download the CUDA Deep Neural Network library (cuDNN) version cuDNN v6.0 (April 27, 2017), for CUDA 8.0 for Mac OS.
- Copy the ‘lib’ and ‘include’ CUDNN folders into the CUDA-8.0 folder
Part 3: Install TensorFlow
TensorFlow handles numerical computation using data flow graphs. We’ll be installing TensorFlow with native PIP.
Install PIP3
- PIP installs and manages software packages in Python. Since we’re running Python 3.3+, you’ll need PIP3 version 8.1 or higher. *This is a list of the packages that PIP will install or upgrade: REQUIRED_PACKAGES section of setup.py
- PIP should have already been installed with Python. Check to confirm version:
pip3 -V
Expected output:
pip 9.0.1 from /Users/'username'/anaconda3/lib/python3.6/site-packages (python 3.6)
3. If you need to upgrade to 8.1+:
$ sudo easy_install — upgrade pip
$ sudo easy_install — upgrade six
Install TensorFlow with PIP3
1. Use PIP3 to install TensorFlow from the Python package:
pip3 install tensorflow
2. Verify the version of TensorFlow installed:
$ pythonimport tensorflow as tf
print(tf.VERSION)
Expected output:
1.4.1
3. If TensorFlow version is older than 1.4.0, you’ll need to upgrade it. This upgrades TensorFlow to the most current version (1.1.4 in this case):
pip3 install --upgrade tensorflow
*Note: As an alternate, you could also use Conda to install:
conda create -n tensorflow python 3.6.3conda install tensorflow
4. Once the installation complete, you can test that it was successful by launching python and running a short TensorFlow test inside the python shell:
$ pythonimport tensorflow as tfhello = tf.constant(‘Hello, TensorFlow!’)sess = tf.Session()print(sess.run(hello))
If the system outputs the following, then you are ready to begin writing TensorFlow programs:
Hello, TensorFlow!
5. *Note — Troubleshooting Binary errors: If you’re running into any errors, there are quite a few cases covered here in Common Installation Issues. You may need to download the specific version of TensorFlow then install it:
$ pip3 install — ignore-installed — upgrade /Users/'username'/tensorflow-1.4.1-cp36-cp36m-macosx_10_13_x86_64.whl
Run the TensorFlow test prompt again to confirm it worked. Thankfully we just have to do all that just once
Now let’s move on to running the environment. 😎
Part 4: The Unity Environment
Summary: You’ll install the Unity plugin, build the Unity environment for the Balance Ball demo, and train the demo project. Then you’ll save the training scene and verify training. You can follow along in the Getting started with Balance Ball Unity docs. I’ll note some small tips below that might help.
Start here with cloning the repo for Unity ML-Agents:
git clone https://github.com/Unity-Technologies/ml-agents.git
Set up the Unity Environment
- Make sure you have Unity 2017.3+ installed. I usually download and make a copy of the repository for good measure too. Note that I’m using ML-agents version 0.3.0b in this tutorial.
- From the Unity startup menu, select Open project. Select the unity-environment folder in:
ML-Agents > ml-agents-master > unity-environment
- When the project loads, navigate to the folder
Assets/ML-Agents/Examples/3DBall/
and open the scene.
Set up TensorflowSharp in Unity
- Download the TensorFlowSharp plugin and double click to import into Unity. It will install into the folder:
Assets > ML-Agents > Plugins > Computer
- Set the Player Settings in:
Edit > Project Settings > Player
- Select platform
PC, Mac, and Linux Standalone > Other Settings panel> Configuration section
- Set Scripting Runtime Version to Experimental (.NET 4.6 Equivalent)
- In Scripting Defined Symbols, add the flag ENABLE_TENSORFLOW - In the
Resolution and Presentation panel
:
- Run in Background should be checked on.
- Display Resolution Dialog should be disabled.
Part 5: Training the Agents
By default, the untrained models will look something like this. Balls will drop and the platforms don’t know how to move unless you attempt to move them manually (not exactly efficient):
- In the Unity Hierarchy panel, expand the
Ball3DAcademy
GameObject and locate its child objectBall3DBrain
. Set Brain Type to External. Using the external brain will train the agents according to our Python package. - Build and save the training environment:
File > Build settings > Build
- Ensure that scene ‘3D Ball
’ is checked.
- Name the file and save it in the python root folder:ml-agents/python/’your_environment_name.app’
- In Terminal, run the training environment:
First, use PIP to install docopt (you’ll need this to run the script in the next step.)
pip3 install docopt
Now navigate to your ml-agents folder:
cd /users/'username'/'path_to_folder'/ml-agents
And then run the script learn.py from python/learn.py
with your environment that you saved previously, also in the python folder <’your_environment_name.app’>
and tell it to--train
python3 python/learn.py python/'your_environment_name.app' --train
See other options for learn.py here: https://github.com/Unity-Technologies/ml-agents/blob/master/python/learn.py
4. The scene will open in a new Unity window and the models will begin training. You can alter the number of steps and speed, etc. It should take only a few minutes for the balls to learn. The models are now trained to keep the ball from falling:
5. When the training session is complete the trained model will be saved in:ml-agents/models/ppo/<‘your_environment_name.bytes’>
Note that you could train agents using Jupyter notebook too (optional)
Embedding a trained model into Unity
So now let’s use an Internal brain to put in our trained environment into the scene.
- In the Unity 3DBall scene
Ball3DBrain
object, set the Brain Type to Internal. - Drag the trained output file
<‘your_environment_name.bytes’>
into the Unity folderExamples > 3Dball > TFModels
. Then select it as the Graph Model in the 3DBallBrain inspector window. - Play the scene. It now runs on the the trained environment and should not drop the ball.
Congrats, we made it!
Your environment it set up and you can try running the other demo scenes. Enjoy!
Additional resources
*If you want to learn more about Python and environments:
*More details about ML-Agents: