Latency Is Not An Issue Anymore When Camera Is Enabled

Capstone Project of the Udacity Self-Driving Car Nanodegree Program

I recently graduated Self-Driving Car Engineer Nanodegree Program. It took a while to complete all of the projects but overall I am very satisfied with entire learning experience. I regret that the course was shortened from 3 terms to 2 terms.

In this short article I would like to focus on simulator part of the Capstone Project because I want to help all future students to avoid disappointment and frustration while working on this project.

To make long story short. The idea of the capstone project is to create a few ROS packages that all together can simulate entire software stack for self-driving car. Simulator enables you to receive a video feed from the camera and create a controller within the ROS which actuates the steering wheel, throttle or break to follow predefined way points path. The software may be executed together with Udacity Simulator or within real car called Carla. It is up to you if you want complete both parts of the project or just simulator part. If you feel that you are not able to complete the project at all you may also opt out by submitting anything and you will still be eligible to receive the certificate. This article is about simulator part where the car must successfully drive entire circle around the test track and abide the traffic lights rules.

If you are reading this article most probably you are experiencing serious latency issues especially when you enable the camera checkbox while car uses autonomous mode within the simulator. You can read more about this issue under following links:

• https://github.com/udacity/CarND-Capstone/issues/210
• https://github.com/josehoras/Self-Driving-Car-Nanodegree-Capstone

I wrote this article because I have not found any repository or an article that describes well how to resolve the problem in generic and elegant way. Udacity internal knowledge base does not provide any viable solution as well.

I managed to successfully run entire simulation on two Ubuntu-based setups:

1. combined — ROS and Udacity Simulator run on Intel(R) Core(TM) i5–3360M CPU @ 2.80GHz with no nVidia graphic card,
2. distributed — ROS executed on nVidia Jetson TX1 platform and Udacity Simulator run on Intel(R) Core(TM) i5–3360M CPU @ 2.80GHz Laptop (Note: simulator can only be run on x86/x86_64 systems). Both devices were connected over Wi-Fi .

For the combined setup I used following tools:

• taskset — set or retrieve a process’s CPU affinity,

-c, --cpu-list - specify a numerical list of processors instead of a bit-mask. The list may contain multiple items, separated by comma, and ranges. For example, 0,5,7,9–11.

• cpulimit — limits the CPU usage of a process.

-l, --limit=N - percentage  of  CPU allowed from 1 up. Usually 1 - 100, but can be higher on multi-core CPUs. (mandatory)

For the distributed setup I used following tools:

• taskset — set or retrieve a process’s CPU affinity,
• tcptunnel — TCP port forwarder.

 --local-port=PORT - local port
 --remote-port=PORT - remote port
 --remote-host=HOST - remote host

Here are examples how to use these tools together with ROS and the Simulator:

1. combined

# cpulimit -l 50 taskset -c 0 ./sys_int.x86_64
# taskset -c 1,2,3 roslaunch launch/styx.launch

2. distributed

# tcptunnel --local-port=4567 --remote-port=4567 --remote-host=xxx.xxx.xxx.xxx & cpulimit -l 20 taskset -c 0 ./sys_int.x86_64
# roslaunch launch/styx.launch

Additionally it was necessary to reduce frequency of Waypoint Updater to:

rate = rospy.Rate(10)

and for distributed setup modify the Server code to include ip address:

eventlet.wsgi.server(eventlet.listen((‘xxx.xxx.xxx.xxx’, 4567)), app)

If you want to use this method you may have to come up with your own parameters passed to cpulimit/taskset depending on the target hardware platform capabilities.

I hope this article will make your learning experience with SDCND Capstone Project even more valuable.

Happy Learning!!!