Introduction to the CARLA simulator: training a neural network to control a car (Part 1)

Intro to CARLA + simple controller for collecting data + training a neural network for steer control.

Introduction: the CARLA simulator is a platform for testing out algorithms for autonomous vehicles.
Motivation: I wanted to check out CARLA, build a simple controller for following a predefined path around the track to gather data, and then train a neural network model using that data.
Approach: I used a simple CNN architecture and modeled the steering angle.
Results: the model is able to drive the car around the track by controlling steering.
Project: https://github.com/asap-report/carla/tree/racetrack/PythonClient/racetrack

EDIT: in a more recent blog post (here) I’ve decided to use a much more lightweight environment than CARLA to train a model that I used to drive *faster* than the algorithm used for driving the car for data collection.

CARLA

CARLA is an open-source simulator built on top of the Unreal Engine 4 (UE4) gaming engine, with additional materials and features providing:

• a LIDAR
• a depth map (emulating a stereo camera, e.g.: ZED or Intel’s D435)
• semantic segmentation data
• two (and soon to be three) towns with pedestrians, other cars, traffic lights, speed limits, and realistic layout
• a repository and data for imitation learning
• a repository for testing out reinforcement learning models
• an option of running without display

But more importantly, CARLA is thriving; it has a great community, it’s documented well, and it’s (subjectively) easy to use.

Unreal Engine 4

One of the reasons I became interested in CARLA was the fact that it’s built using UE4. I wanted to hone (or rather: keep alive) my C++ skills, and at the same time tinker with machine learning for autonomous vehicles, so CARLA was the perfect combination of these two worlds.

But something I didn’t anticipate is that UE4 allows for developing quite advanced features without the necessity of implementing them in C++. For that, UE4 has a dedicated visual scripting language called Blueprints. I’ve used Blueprints for the creation of the race track (more details further).

Alternatives to CARLA

There are several open-source simulators for autonomous cars and/or racing, but very few with capabilities comparable to or greater than CARLA’s. Here’s a list of those that I’ve considered (with their respective environments in parentheses):

• Autoware (Gazebo)
• AirSim (UE4 & Unity)
• TORCS (OpenGL)

And two simpler ones:

• the Udacity simulator (Unity)
• Donkeycar simulator (Unity)

In the end, I only considered CARLA and AirSim since I wanted to work with UE4, and chose CARLA because it seemed easier to contact with the community via their discord chat.

The race track

To build the race track I took the Town02 map, deleted all props (as suggested here), and thus prepared the scene for the race track.

The track is composed of RoadMesh elements that look like this:

But to be able to create a smooth track I had to incorporate this element into a blueprint script called Track Generator. The race track is then created like this:

The race track on which I’ve actually rode looks like this:

I wanted to have sections of: tight corners, standard corners, and a couple of long straights. But, full disclosure, I have no idea what makes a track better or worse. For my purposes, this race track was perfect.

An interlude: a tour of the project

I’ve forked the carla-simulator repository, branched off from the stable version (release 0.8.2) into a branch racetrack and created a directory carla/PythonClient/racetrack where I keep all the necessary scripts for reproducing this work.

The two most important scripts are: client_controller.py and train_on_depth.py, where the first one is used for interfacing with CARLA, for controlling the car and for gathering data, whereas the second is used for training neural networks on the collected data.

Before running the client_controller.py script, remember to first initialize CARLA. On my system it requires the following:

cd ~/CARLA
./CarlaUE4.sh /Game/Maps/Racetrack01 -windowed -carla-server

After which, to control the car using a proportion-derivative (PD) controller (more details below), I call:

python3 client_controller.py \
   --speed 45 \
   --controlled_name pd

And the result should look more or less like this:

We’ll get to the train_on_depth.py script in a minute.

The simplest possible controller

Based on the experiences gathered from the Udacity course, I knew one thing: I didn’t want to control the car myself. It would require multiple data collection sessions, would quickly become tedious and time consuming, and I would inevitably make mistakes that would require editing.

To reach my goal (training a neural network capable of going around the track), I started with the simplest controller that would collect the data for me. And it doesn’t get much simpler than the PD controller:

in which the steering angle at a given time t depends on the Cross-Track Error (CTE), and the time derivative of the CTE. The CTE is the distance between the center of mass of the car, and the desired trajectory. Also, the controller needs to be calibrated, i.e. it’s up to us to choose the coefficients kp and kd such that the car drives (relatively) smoothly. (The fact that Medium doesn’t have support for LaTeX is a pain to say the least.)

Admittedly, this is not the simplest possible controller because, as you might have thought, one might try a “P only” controller that ignores the second component. And yes, this is an even simpler controller but the dCTE/dt component plays an important role of stabilizing the output. Without it, the controller quickly becomes erratic:

so it wouldn’t get the car around the track.

Throttle

The steering angle is not the only actuator we have at our disposal, there is also throttle. (And a handbrake, but we’re not going to touch it… yet.) But what we’re actually interested in is not throttle, but the thing it influences — velocity.

In the first iteration of this project I didn’t really care for velocity, I just wanted the car to go forward; not too fast, not too slow. To that end, I used a simple clip_throttle function that keeps the throttle within reasonable bounds, and tries to keep the speed close to the target_speed:

Notice that this results in a wide range of throttle values, which in turn results in high variance of the velocity. But that’s OK; my reasoning was that this “augments” the data set and yields a more generalized model.

Technically, you could take the time and build a separate PD controller for the throttle so that a target speed is kept fairly constant. But this would require its own (kp, kd) parameters, making the model slightly more complicated.

But more importantly, in the second blog post we’ll use a controller that controls both actuators, steering angle and throttle, simultaneously. This approach is called Model Predictive Control (MPC), and is admittedly more complicated, but it provides a trajectory that is much more efficient. And if this didn’t whet your appetite for the second blog post, here’s a video illustrating how an MPC-controlled car drives around the track:

Right away you can notice that the frame rate is much lower. This is because MPC requires some heavy optimization to be done on the fly, and my implementation of this controller is written entirely in Python.

Building a neural network

Partly inspired by this blog post and partly dissatisfied with the performance of a neural network model trained on pure RGB input (done as part of the Udacity Self-Driving Car Nanodegree), I’ve decided to train a model on the depth map data that can be easily collected in CARLA. This is how the world looks like when seen through the depth map lens:

Admittedly, it’s pretty pixelated and a bit unclear. But note that this representation carries 3D information that is essential for driving the car along the track.

I used keras for training the neural network, and used a fixed, singular architecture. I prefer raw code instead of fancy visualization tools, so here’s the definition of the model:

I didn’t spend much time tweaking the architecture or the optimization parameters. I chose “reasonable” hyperparameters, and instead focused on modifying the input and adding auxiliary losses. But more on that in the second blog post.

The model had one job: predict the steering angle for a given depth map frame. It didn’t make much sense to predict throttle because it was independent of what was in front of the car (at least for our PD controller; this will change in the second blog post). So the throttle was also modulated using the clip_throttle function.

To train the model, I ran:

python3 train_on_depth.py -c pd

I had about 90k frames and the net took about 2h to train, but that’s because: 1) the model was over-the-top-complex, 2) I didn’t shrink the input image, and 3) the default number of epochs was 50. However, the MSE on the test set quickly became flat:

And in reality, starting with “Epoch 2” the scatter plot of predicted vs actual steering angles looks pretty much constant:

This is how the model with the lowest MSE on test set managed to drive around the track:

I think the coolest thing is that the neural network model seems to stabilize the oscillations better than the PD controller.

Problems

The biggest problem I haven’t yet solved is model selection. The MSE was insufficient, because it wasn’t always the case that the model with the lowest MSE on the validation set was the best. One way of dealing with this is to validate the model automatically in CARLA, without display. But I would like to have a more general approach, one that I could apply in the real world.

Also, I’ve noticed that models with the lowest MSE failed to steer the car out of tight turns. That is, high (but rare) values of the steering angle were never produced by such models in the simulator (even if on the test set the model seemed capable of yielding those steering angles). So I’m inclined to evaluate the performance of the model in two types of scenarios: normal, and extreme situations. But that’s for the future.

Introduction to the CARLA simulator: training a neural network to control a car (Part 2)

Be sure to check out the second part of this series. It has more content (an euphemism for: “it’s longer”), and takes a look at different sources of data, created by different controllers, including a game pad!

Training a neural network for driving an autonomous RC car

There’s also an “unofficial” Part 3 which utilizes the exact same machine learning pipeline used for this project, but the neural net is steering an actual mobile robot and the racetrack is a path around a pond that’s next to my house.

References

Unreal Engine 4: this course on Udemy is a great intro but only if you’re into C++. Also, it’s recommended by the creators of CARLA.

Race track: For building the race track I followed this tutorial: youtube link. Also, I found this part from the UE4 documentation valuable. Actually, the whole UE4 is documented very well, here’s one of the gems I intend to explore.

If you enjoyed this post, please hit the clap button below and follow our publication for more interesting articles about ML & AI.