Try Unity ML-Agents, New feature to connect Unity & TensorFlow : Imitation Learning
Unity Machine Learning Agents (ML-Agents) is an open-source Unity plugin that enables games and simulations to serve as environments for training intelligent agents.To know Unity ML Agents Workflow, watch video below.
And ML-Agents v0.3 Beta released and imitation learning was newly supported. So I started it with Unity. In this article, I try to show you how it works using prepared samples. If you want to know about ML-Agents more, see this link first.
Environment
・Windows10(64-bit)
・Unity 2018.1.0f2 (64-bit)
・ML-Agents Beta 0.3.1b
・Miniconda3
Installing ML-Agents for Windows
If you don’t install ML-Agents yet , see this page and install it. I installed Miniconda instead of Anaconda.
Build a sample of Imitation Learning
Step1: Open project of ML-Agents with Unity. Project path is ml-agents\unity-environment. Perhaps a warning dialog will appear, but there is no problem(Go ahead).
Step2: Open the imitation learning sample scene which name is BananaImitation. It's path is Assets\ML-Agents\Examples\BananaCollectors
Step3: Select File > Build Settings… And click Build to create your build. I created directory named BananaImitation for build. The path is ml-agents\BananaImitation
Step4: If build completed with a result of ‘Succeeded’ , make sure BananaImitation.exe is in the output directory.
Play game, and teach agents how to move
To launch the training process , run the following commands in an Anaconda Prompt (Be sure to activate the ml-agents Conda environment by typing activate ml-agents):
python python/learn.py <env_name> --train --slow
<env_name> is the path to your built Unity executable. In my case, BananaImitation\BananaImitation.exe
After successfully launching the training environment, Play game with WASD key, and earn more yellow bananas. Student agents will learn how to move, and how to earn yellow bananas.
After a few minutes, the game will end. Trained model is output to ml-agents\models\ppo. In my case, model name is BananaImitation_ppo.bytes.
Embed a trained model into the Unity project
Step1: Move trained model into the Unity project. For example, Assets\ML-Agents\Examples\BananaCollectors\TFModels
Step2: Download Unity TensorFlow Plugin (TFSharpPlugin.unitypackage ) from this page, and import it to the Unity project.
Step3: In Unity Editor, Go to Edit -> Player Settings and add ENABLE_TENSORFLOW to the Scripting Define Symbols.
Step4: Select gameobject named StudentBrain in the BananaImitation scene (StudentBrain is child of Academy) .
Step5: In the Inspector window, set Internalto Brain Type, and set trained model to Graph Model.
Play scene, and observe how Student agents work
Click Play , and Student agents make decisions from brain.
Student agents earn bananas, but soon their performance decrease.
Improve training
To improve Student agents’s performance, change the Brain Parameter. The name of the parameter to change is Stacked Vectors, change it from 3 to 20. Don’t forget to change both TeacherBrain and StudentBrain.
When the parameter change is completed, build project, train student, and re-import trained model. And play scene, observe how Student agents work once again.
Looks Good :)
Conclusion
It is amazing to grow like this in a short time!
If you are interested in, Let’s try imitation learning with Unity.
Reference
ML-Agents v0.3 Beta released: Imitation Learning, feedback-driven features, and more(Unity Blog)
