This will be a quick story on how to install the ML-agent environment and run the 3DBall example of the unity ML-agents github repo. I wanted to start trying my hands at reinforcement learning and figured the ML-agents repo of Unity was a good place to start.
The installation guide is pretty clear and well done, but it only goes into depth on how to set everything up on Windows or OSX. I personally don’t have an OSX machine capable of any kind of serious machine learning and Tensorflow no longer supports GPU accelerated training on mac. I also don’t really want to setup and maintain a development environment on Windows. No, my preference is to use Linux to do machine learning experiments but unfortunately the ML-Agents installation guide does not really explain how to set this up. I did manage to get it working in the end and it wasn’t too much of a hassle but I hope maybe I can save someone else some time.
Note: The Linux version I use is Ubuntu 16.04 your mileage on other versions may vary.
Step 1: Install Unity
Unity is not yet fully supported and functional on Linux but a Beta version already exists which has all the functionality needed to experiment with ML-Agents. To install unity on Ubuntu download the latest version of the linux download assistant from here (just go to the last post on the last page). Once that finishes downloading open terminal and navigate to the Downloads directory. Once there run the following command (you might need to adjust the filename to the specific version you downloaded):
sudo chmod +x UnitySetup-2018.2.3f1
After that a simple execution of the file will start the installer:
At this point just follow the steps in the installer, I found I didn’t need to change anything and could just keep pressing forward.
Step 2: Setup ML-agents
Setting up the ml-agents repo is pretty simple, clone the git repo using the following command:
git clone https://github.com/Unity-Technologies/ml-agents.git
And then navigate to the python folder inside the cloned repo and using Python 3.5 or 3.6 run the following command:
pip3 install .
Step 3: Run the example
At this point everything is setup and we can follow the steps in the basic guide, the first problem however is that using the pretrained model with the Tensorflow Sharp plugin the editor crashes. I am not sure exactly why this happens but you don’t need it as you can do training and inference using the python interface. Skipping the pretrained model the following steps should result in a model being trained with Tensorflow. Keep in mind these steps are pretty much directly copied from here so if you get stuck maybe look over there as they did a slightly more elaborate explanation with supporting pictures.
- Launch Unity
- On the Projects dialog, choose the Open option at the top of the window.
- Using the file dialog that opens, locate the
unity-environmentfolder within the the ML-Agents toolkit project and click Open.
- Go to Edit > Project Settings > Player
- For each of the platforms you target (PC, Mac and Linux Standalone, iOS or Android):
- Option the Other Settings section.
- Select Scripting Runtime Version to Experimental (.NET 4.6 Equivalent or .NET 4.x Equivalent)
- In Scripting Defined Symbols, add the flag
ENABLE_TENSORFLOW. After typing in the flag name, press Enter.
- Go to File > Save Project
- In the Project window, go to
Assets/ML-Agents/Examples/3DBallfolder and open the
- In the Hierarchy window, select the Ball3DBrain child under the Ball3DAcademy GameObject to view its properties in the Inspector window.
- On the Ball3DBrain object’s Brain component, change the Brain Type to External.
- Open a command or terminal window.
- Nagivate to the folder where you installed the ML-Agents toolkit.
- Change to the
python3 learn.py --run-id="test" --train
- When the message “Start training by pressing the Play button in the Unity Editor” is displayed on the screen, you can press the play button in Unity to start training in the Editor.
At this point you should see a bunch of platforms moving very erratically in the editor and slowly get better at balancing a ball on them. If that is the case then success! Now you can start experimenting with reinforcement learning.
Note: When using the external brain I find that the main camera frame is no longer centered in the window (only bottom right corner visible). By switching the layout using window->layout->2 by 3 I got a working camera view.