Reinforcement Learning for Home Robotics pt. 2
Building the Environment
Introduction
In this article we’ll take our first step towards building the RL system outlined in the above image. Here we’ll be building the environment that our purposed home robot will exist in. In an attempt to not bite off more than we can chew, the world will be fairly limited with the following restraints:
- All objects in the world are static.
- We use the same layout for each episode.
- The commands given to the robot can be random for each episode.
Our world will be based on the home layout introduced in the previous article, which is shown below:
Our goal for this article will be to build this static environment in ROS Kinetic using Gazebo.
Note: I am aware that ROS 2.0 is on its way, but as of this writing it is not officially released yet. Furthermore, I haven’t read up enough on it to be comfortable converting all my current work over to platform. When its officially released and I’ve taken the time to brush up on it I’ll write up some articles for the blog on how to convert things over to the new ROS 2.0 system. Until then, I’ll be assuming the current ROS 1.0 system, of which I generally use Kinetic on my laptop.
Gazebo Building Editor
We’ll start by putting together the layout for the Agent’s environment using the Gazebo Building Editor.
Above we can see the Gazebo Building editor. As this is not a comprehensive tutorial on the features of Gazebo, we’ll only be covering the features needed to build the training environment for Marvin. If you would like more information on the Gazebo Building editor you can find it here.
Our first step in designing the environment is to our import the layout. We can do that by clicking the “import” button on the bottom left of the window. From here we can select the png file that represents our layout (in this case I’ve used the same png file presented as “Marvin’s World” earlier in this article).
Once we’ve imported the layout, we’ll be prompted to set a scale. We do this by selecting an interval on the image to represent one metre. In this case I’ve used roughly the length of the doorways to represent that value.
Next up is to place the buildings walls. Walls can be placed in the model by selecting the “wall” icon on the left side of the interface and then tracing over the walls in the image template that was imported in the previous step.
For our final step in using the building editor, we will add texture to the building. To do this we select the “wood” texture (again from the left side of the interface) and apply it to the walls by clicking on each them in the 3D preview of the bottom portion of the interface.
Gazebo World
Once we’ve finished our building, we’ll want to place it in a Gazebo world. This is fairly easy as after we leave the building editor it will be pre-placed into the world for us. The first thing we’ll want to do is change the position of the building so the (0,0) position (marked by the blue z-axis in the image) is in the position where we want Marvin to start.
Next our Gazebo world needs some people (at least if we are going to follow the design provided). Gazebo has some built in models that can help with this. We’ll grab one of those models from the Insert tab called “standing person” and insert it everywhere that we have a “stick person” on our original floor map.
Finally we need to add our soda can, which we’ll find is fairly easy to do in Gazebo, since it provides us with a “soda can” model that matches exactly what we are looking for! We’ll place that in the analogous position on in the world and then save the file.
Once all this is done, our world should resemble the one shown below:
Launching the World
Our final step in setting up the environment for our agent will be to create a launch file for it in ROS. This will make it easier for us to configure the simulation later as we layer on more components.
Below is the launch file for loading the RL Environment. This launch file loads Marvin’s robot description file, loads the world in gazebo and places the robot at the 0,0 position in the world.
<?xml version="1.0" encoding="UTF-8"?>
<launch>
<include file="$(find rover_slam)/launch/marvin_bot_description.launch"/>
<arg name="world" default="empty"/>
<arg name="paused" default="false"/>
<arg name="use_sim_time" default="true"/>
<arg name="gui" default="true"/>
<arg name="headless" default="false"/>
<arg name="debug" default="false"/> <include file="$(find gazebo_ros)/launch/empty_world.launch">
<arg name="world_name" value="$(find rover_slam)/worlds/RL_World.world"/>
<arg name="paused" value="$(arg paused)"/>
<arg name="use_sim_time" value="$(arg use_sim_time)"/>
<arg name="gui" value="$(arg gui)"/>
<arg name="headless" value="$(arg headless)"/>
<arg name="debug" value="$(arg debug)"/>
</include>
<!--spawn a robot in gazebo world-->
<node name="urdf_spawner" pkg="gazebo_ros" type="spawn_model" respawn="false" output="screen" args="-urdf -param robot_description -model rover_slam" />
</launch>
The above code snippet assumes that the world file has been named “RL_World.world”, a urdf file loaded by a launch file named “marvin_bot_description.launch”. It should be placed in the customary “World” folder of the ROS package it used with.
Next Steps
That concludes what needs to be covered for this article. As creating an environment in Gazebo is fairly straightforward, this entry turned out to be a bit shorter than the others have been so far (and likely than any of the ones that come after it).
Coming in the next article I’ll be introducing modifications to the current simulation of the Marvin robot described so far in this blog. These modifications are designed to facilitate the robot being used in a RL context rather than depending on a hard coded set of rules modeled after the subsumption architecture.
Until then,
Share and Enjoy!
