Reinforcement learning with Unity Part 3.1— Making a cube and moving it around
Hi everyone, welcome back to the third part of my journey to implementing Reinforcement learning using Unity. In the previous article we finally finished setting up Unity and VS Code! In this article we are going to make a simple cube and control it with our keyboard. Let’s get started!
Background 1 — Unity project structure
Unity has a specific project structure we have to work with. When we open up a project for the first time what we see is an empty Scene. A scene, as the name suggests, is an environment where just one kind of scenario is happening. For example, in a game there could be separate scenes for a menu page, options page, actual game page and so on. For us a scene will be like a workspace where we will have a cube moving around on a plane. So without a scene (or a workspace) we cannot make anything and that’s why by default Unity project is initialised with an empty scene.
If we now look at the file system structure (open a folder and navigate to where the project is) we can see that there are many folders in “OurFirstProject” folder. I am not going to drag on to explain what each folders are for but the most important folder for us right is the “Assets” folder. In this folder we are going to have all of our important things like our scripts, prefabs, models, etc so make sure you remember this. By default though, we only have one folder named “Scenes” which of course, contains our default scene at the moment.
Background 2 — Unity’s interface
You can read in more detail about each section of Unity’s interface in the official documentation. But quickly,
- On the top toolbar with play button is where we control the mode — either editing mode or game mode.
- On the left hand side pane where it says “Hierarchy” we will be able to see all the objects in the scene in a hierarchical view.
- In the middle we have “Scene” and “Game” tabs. This is where we will be able to see the object in a “scene” view which is what we see while we are editing and in a “game” view which is what the actual rendered game would look like.
- On the right hand side pane where it says “Inspector” we will be able to see all of the information about an object when we click on the object on the “Hierarchy” pane.
- On the bottom pane where there are 2 tabs — “Project” and “Console”, this is where we can navigate through the Assets folder in our project, and see all the displayed messaged in console, respectively.
Background 3 — Why not implement reinforcement learning straight away?
From my experience it is very important that we check the system (in our case the object) moves and behaves in the ways we expect it to move and behave. The best way and the easiest way to do this is by controlling the system manually. Later we will see that we can visually verify the reward function whilst controlling the system manually. But for now let’s not worry about what the reward is or in fact what reinforcement learning even is!
I apologise for boring you with all of these background stuff but they are very important so I had no choice but to go through it. Anyways, now we have gone over the boring stuff let’s get to the fun stuff!
Let’s make a ground plane!
*Before we begin, we should note that the units used in Unity follow the metric system, i.e. metres for positions, degrees for angles, m/s for velocities and rad/s for angular velocities, etc.
If you have any experience in making a 3D object using 3D modelling software like Blender then this will be pretty much exactly the same. Otherwise do not worry, it will be super easy!
In the “Hierarchy” pane, right click then “3D Object” -> “Plane”.
That’s it! super easy as I said.
Modify position
By default, any object we create will be created in the centre point of our viewpoint which means it may not be in the our origin (x, y, z coordinates of 0, 0, 0). So let’s set our plane’s position to origin:
- Click on the Plane.
- Right click on “Transform” on the right hand side pane called “Inspector”.
- Select “Reset”.
You should’ve seen the values in “Position” changing to (0, 0, 0).
*Note — each object will have a predefined dimension by Unity so for a Plane default size is 10 m x 10 m x 10 m. For a Cube is 1 m x 1 m x 1 m.
Let’s make a cube!
Just like making a plane, in the “Hierarchy” pane, right click then “3D Object” -> “Cube”.
Let’s modify the cube!
Since the cube has its own coordinate system originating from its centre, there is an offset of 50 cm in every direction. What this means is that if we had a plane as a ground at origin, then the cube will be half-way sunk into the ground. So we need to add that offset of 0.5 in the y-direction to bring the cube up to the plane’s level.
Modify the scale
So right now our cube’s size is 1 m x 1 m x 1 m. I think it’s just a little to big so I am going to change it to 50 cm each. I can do this by changing the values in “Scale” under “Transform” in the Inspector panel to 0.5 each.
This means we have to change the offset in the y-direction again to 0.25.
More advanced topic — different coordinate systems (you can skip this part)
In Unity we have different axes for each object we create — we also call them “frames” or “coordinate systems”. To elaborate, the coordinate system of the scene is what we call “global” or “world” coordinate system (we can also call it global or world frame).
When we create a cube, the cube has its own coordinate system originating from its centre. So when we talk about it from the cube’s perspective we can say that we are in the cube’s local coordinate system or local frame which is of course different to the global coordinate system. So later we will see that an object will have its “position” and its “local position” where the position is in the global coordinate system and the local position is in the object’s original local coordinate system.
So when we create a cube and then set its position to the origin (0, 0, 0), what we are doing is we are aligning the cube’s local coordinate system’s origin to our global coordinate system’s origin.
Another thing to note is that when we create a cube with the size of 1 m x 1 m x 1 m, the cube extends 50 cm from its origin of the local coordinate system in every direction to make 1 m length on each side.
Unity’s physics engine
You might be wondering — how do we actually control the cube to move on the plane though? That is a very good question! We could move the cube by changing the position of the cube every frame with key stroke in each direction or maybe also move it by assigning some sort of acceleration to each key and the duration of the key press?
I mean… we could? but that involves way too much maths and physics — we definitely want to avoid as much maths and physics as possible if we can, because just even thinking about it gives me a headache.
So that’s where Unity’s “Rigidbody” comes into play. Rigidbody lets us control the object through the Unity’s physics engine. If the Rigidbody is attached to an object the object will automatically be affected by gravity and collision. We can also code to apply different forces to the object to move it. You can visit the documentation to find out more about it but we will move on for now.
Attach Rigidbody to the Cube
On the left “Heirarchy” pane, click on the Cube object. Then on the right “Inspector” pane at the bottom click on the “Add Component” button. Search for “Rigid body” and select “Rigidbody” and not “Rigidbody 2D”.
That’s all there is to attaching the rigid body and therefore, attaching a physics engine!
That’s it!
I think this article is going to be super long if I include coding section to this so I will divide this up.
I hope this article gave you a little bit of more hands-on experience with Unity and their interface. I apologise again if this was a little too boring but I had to cover it, I promise the next article will be more fun and we will get the cube to move around!
Thank you for reading this article and hopefully this was helpful!
