Using TurtleBot in Deep Reinforcement Learning
In a previous set of tutorial articles (Part 1 of the tutorial), I explored an implementation of TD3 network architecture in combination with ROS simulator for learning how to navigate a mobile robot from direct sensor inputs. While the network architecture and the simulator are (in theory) robot model agnostic, it did use a specific Pioneer 3DX mobile robot platform. There are certain implications that come with using a specific robot model. Mainly, the inferred kinematics. The learned actions are dependent on the expectation of how will they be executed. There is a learned understanding, that making a certain motion will result in a specific state of the environment. This, of course, depends on such parameters as the robot’s length, width, wheel diameter, wheel location, etc. That means that, if you learn a motion policy with one type of robot model, it is not trivial to transfer this policy to a different type of robot with different sizes and wheel diameters. While coefficients and strange mathematical equations can be found to do this, it is much better and simpler to just learn the motion policy with the same robot platform in simulation as will be used in real life. Unfortunately, Pioneer 3DX is already quite an old mobile robot platform and is not popular with many researchers anymore. (The can of worms that is getting ROSARIA to work is not helping either). Therefore, probably one of the most frequent questions I get asked is “How do I use TurtleBot instead?”. So here I will explain how to use TurtleBot model in learning mobile robot navigation policy through our Deep Reinforcement Learning pipeline.
Luckily, not a lot of changes have to be made when we want to use a different robot platform. So, regarding the programmatic implementation of the network architecture and how it works together with ROS simulator, feel free to refer to the aforementioned tutorial — Part 1. Almost the only thing that will change is the call to the robot model when launching the simulator. Here, to describe the implementation, I will use this GitHub repository and specifically the Noetic-Turtlebot branch:
For the installation guide, please follow the commands in the Installation section of the repository.
The nice people from ROBOTIS have provided the turtlebot 3 models for ROS on their GitHub that we will be using in our implementation.
Calling The Robot Model Launch
After successfully cloning and compiling the repository, we should have all our required files set up, including a directory consisting of select folders and files from the original Turtlebot 3 repository.
When starting the training in ROS simulator, the first thing the program does is execute the launch file. The launch file is located in:
DRL-robot-navigation/TD3/assets/in file multi_robot_scenario.launch
This file takes care of launching the gazebo simulator and its world in line 7. In line 18 we launch the Rviz node with the specified configuration. So in between, starting from line 11, is where we launch our robot model. With the <include> tag, we specify additional launch fines that we want to launch when launching our training. In this case, we are specifying our turtlebot3 model launch file. We have set a couple of arguments, that we might want to specify when launching the selected model.
<arg name="model" value="waffle" />The turtlebot3 repository has provided implementation for 3 types of turtlebot robots. They are — burger, waffle, and waffle_pi. To specify, which robot model we want to use in our simulation, we can simply set the argument with the name “model” to a value with the type of robot we want to use.
<arg name="multi_robot_name" value="r1"/>The “multi_robot_name” argument with set the name used for this specific robot instance.
<arg name="robot_position" value="-x 0.0 -y 0.0 -z 0.01 -R 0 -P 0 -Y +0.0" />“robot_position” will set the location where the robot will be spawned for the first time when starting the simulator.
Launching the Robot Model
From the path in the call to robot model launch, we see that we are calling a turtlebot3_model.launch file, located in:
DRL-robot-navigation/catkin_ws/src/turtlebot3/turtlebot3_bringup/launch/First, we have specified the arguments and their default values that we would need to launch the file (we set their real values in the multi_robot_scenario.launch file).
Then we set the parameter of the robot description that we will actually use in the simulator in line 7. We specify the path to the xacro file of the robot description. Notice that we are using the file:
turtlebot3_$(arg model).urdf.xacroHere, the $(arg model) is an argument that we had set in the multi_robot_scenario.launch file. If we set the robot model to waffle, our robot description will call a file with the name:
turtlebot3_waffle.urdf.xacroThese robot description parameters are then passed into the urdf spawner node in line 10, which will take care of adding our model to the simulator. Afterward, we start the robot state publisher and robot joint publisher nodes that will take care of keeping track of different connections and joint transforms of our robot.
Robot Description
The robot description file (if calling a waffle type of robot) is located here:
DRL-robot-navigation/catkin_ws/src/turtlebot3/turtlebot3_description/urdf/turtlebot3_waffle.urdf.xacroThis file specifies the robot construction and adds the sensors to the robot model. The DRL method used for obtaining motion policy is based on sensor data from Velodyne Puck 16-channel LiDAR. This is not originally included with the turtlebot robot in the original repository. Luckily, adding this sensor is very easy. Simply include the sensor in this xacro file.
The <origin> tag will specify the location of the sensor in relation to the parent link (in this case “base_link”). Here, we are placing the sensor at a location where a laser scan was located in the original turtlebot repository. That means that the height of the velodyne puck is different than on our original Pioneer 3DX model. This brings us to the only code change. When obtaining data from the Velodyne LiDAR, we filtered out points that are 20 cm lower than the plane of the sensor. By placing the sensor at a different height, we also need to adjust this value. Simply, set the appropriate height value in:
DRL-robot-navigation/TD3/velodyne_env.pyBy setting the filtering parameter according to the height of the sensor on your robot model:
And that is it! This should allow you to use and modify the turtlebot robot model call stack. If there are any questions, do not hesitate to open an issue in the repository.
Quick Note
For the mobile robot platform to work in ROS Noetic, I also slightly modified the gazebo files (waffle robot file for reference):
DRL-robot-navigation/catkin_ws/src/turtlebot3/turtlebot3_description/urdf/turtlebot3_waffle.gazebo.xacroChanges were made in the differential drive plugin:
The following tags were inserted into the plugin:
