Why do we need Open Source Self Driving Car development and how to get started

Self Driving Cars (SDC) news are all around us, and soon (next 2–5 years) we will see lots of SDC on the roads, and they will be driving us!

We see announcements almost every day about established (auto) companies or startups that started to work on technologies related with SDC.

Although there is publicly available robotics research and open source projects like Robotics Operating System (ROS robotics in general) and Autoware (SDC specific), there is still a need for more people to get involved and do research, development and testing in this area, to ensure that we will get safe SDC on the roads in the next few years.

Also getting involved in OS SDC development is a very good way to learn about autonomous mobile robotics (very important area of robotics for our future mobility and personal assistants “robots” capabilities), SDC being just an outdoor autonomous robot with people on board and around, and the technologies used here can be applied in many other cases, in constrained or unconstrained environments.

This is the continuation of my (quick) deep dive into Open Source (OS) SDC development SDC From 0 to 60 (*) in 4 weeks as part of Udacity OS SDC challenges (that I posted a week ago), I suggest you read that one first to get a feel of what it means to do OS SDC development.

I hope I convinced you already :-) so let’s get started.

In the next few weeks I will show you how to get the results I present in the deep dive article and much more, bellow is the first part in this series regarding preparing the development environment and testing it for readiness.

First of all, here are a few required things to ensure a smooth “SDC” ride:

• a computer, preferably a recent (Linux) desktop with a Nvidia GPU (required for deep learning and 3D simulators) with compute capability > 3, for example my current machine configuration is this and this will soon be my son’s deep gaming/learning/virtual reality machine, with partial Series A round funding from me :-)
• a bit of Linux experience, as most robotics software is running only/best on Linux
• a bit of Python experience, as it is used a lot in robotics and AI/machine learning (ML) and especially to build deep neural networks (DNN)
• and lots of curiosity, patience and perseverance (in order to achieve good results)

In the past 5 weeks I recommended and helped people to use Docker, it is the easiest way to get started and to keep your host machine clean while trying a lot of software in docker containers, in some cases is the only way to run the required software (because of different OS dependency).

Install Docker following the instructions here:

https://docs.docker.com/engine/installation/linux/ubuntulinux/

Install nvidia-docker (on Linux only, this will give you GPU acceleration in the Docker containers) from here:

https://github.com/NVIDIA/nvidia-docker

Most of the instructions bellow will target Ubuntu 16.04 host and use nvidia-docker, if you don’t have a Linux host, you can still install Docker and use the instructions on Windows and OSX, but you may run into some problems, also GPU acceleration will not work on Windows and OSX, so it may take days to train DNN models on CPU, while a GPU can do in hours.

The Docker installation instructions for Windows and OSX are available here:

https://docs.docker.com/docker-for-windows/
https://docs.docker.com/docker-for-mac/

Now that we have Docker and nvidia-docker (this is only on Linux) installed we can go ahead and pull my Docker image that contains a lot of software ready to use for OS SDC development, make sure you have at least 20-30GB available in the root file system (check in Terminal with “df /”), as the docker image is about 17GB uncompressed (the compressed/download size is 8GB) and you’ll need (lots of) space also for datasets (which are pretty big), then run in a Terminal:

docker pull gtarobotics/udacity-sdc

While this is downloading lets get some code that we will need later, open a new Terminal and run these commands to get my self-driving-car project from GitHub:

mkdir ~/sharefolder
cd ~/sharefolder/
git clone https://github.com/gtarobotics/self-driving-car
cd self-driving-car
chmod 755 *.sh

After 5–15 min depending on your Internet connection, the Docker image should have been pulled/downloaded and be ready to use.

Now we can launch the Docker image by running one of these scripts.

In GPU mode:

./run_nvidia_docker-sdc-ros-gpu.sh

or CPU mode:

./run_nvidia_docker-sdc-ros-cpu.sh

On Mac OSX run this script instead:

./run_docker_sdc_ros_cpu-on-OSX.sh

At this point we should see the Docker container console prompt, and we should be running as root (you can ignore for now the vnc messages we will get to them later, you can also do CRTL-C to stop noVNC).

First lets quickly test if GPU acceleration works by running the following command:

./run_quick_benchmark.sh

If you see this kind of output:

Step 400 (epoch 0.47), 5.1 ms
Minibatch loss: 3.217, learning rate: 0.010000
Minibatch error: 9.4%
Validation error: 2.8%

Than you have a pretty good GPU and it is used properly, if the Step time is over 15 ms you probably have an older GPU with less CUDA cores, less memory and lower running frequency, if you see values over 100 ms you probably run in CPU, because you don’t have a supported GPU and you started docker using the CPU script.

Lets try one more quick test, to see if the UI/OpenCV is working by running the following commands:

cd ~/sharefolder
curl https://archive.org/download/NASA_Launchpad_MSL_HD/NASA_Launchpad_MSL_HD.mp4 -o NASA_Launchpad_MSL_HD.mp4 -L
python3 self-driving-car/docker-instance-scripts/test_opencv.py -v NASA_Launchpad_MSL_HD.mp4

You should see a window like this and the video will playback in gray scale (at faster speed than normal):

At this point we know that python3, Tensorflow with GPU acceleration (or not) and OpenCV is working, so we are ready to go further, to download datasets and analyze them.

To be continued …