Self-Driving car on a Hiking Trail

Introduction

Self-driving cars on urban streets make effective use of road markers, traffic signs, and various sensors to stay on the road. The roads are generally well lit and maintained, to make day-to-day traffic as smooth as possible. Our project aims to explore what can be done in more chaotic conditions, without explicit road markings and armed with only a camera. More specifically, we attempt to teach a robot platform to navigate narrow footpaths in the wild using computer vision and neural networks.

The DonkeyCar platform

The chosen platform is DonkeyCar - a raspberry pi based robot platform aimed at developing self-driving cars using neural networks and computer vision. The platform comes with several convenient software utilities to get up and running as soon as possible, as well as a few predefined neural network architectures to get started with. The stock DonkeyCar platform was modified by adding a 3S LiPo battery for longer runtime and an Xbox controller receiver for controlling the robot remotely with low latency.

DonkeyCar set up for capturing training data.

First models

To prove the concept of driving on the desired type of trail, the first model was trained on a short track to give it a better chance of success. To collect the training data, the track was driven several times in both directions. As the track was curved, it allowed us to check whether steering works correctly in both directions, thus giving very good visual feedback and showing whether the model behaves expectedly.

DonkeyCar being controlled manually to capture training data.

DonkeyCar has few already implemented models, our first experiment was using the default model “Keras Linear”. Like most of the models implemented, it takes the camera image as an argument and feeds it through five convolutional layers followed by 0.2 dropout and flattens the output. In the linear model, this is followed by two feed-forward layers (100 and 50 neurons each, followed by dropout). The model outputs two values: one for steering and one for throttle.

According to the DonkeyCar website, it runs well on a Raspberry Pi, is quite robust, steers smoothly and our results support that. The results with the simple model were satisfying and during around six test runs it did not make any errors on the track it was trained on. If anything, it drove even slightly better than when it was controlled manually.

DonkeyCar driving autonomously on the track that it knows well.

The same model was then tested on a similar track, but with the surrounding environment slightly changed. During this test, the car showed a significant amount of correct behavior but also had some errors. It was clear that when the driving direction already went a bit off, then the model was not able to make the necessary corrections.

DonkeyCar driving on a road it hasn’t seen before.

Further testing verified that issue and it can be explained by not having training data that demonstrates how to act when the car is already going off the track. This data is more difficult to gather as well, as it requires creating this situation enough times without recording the creation of the situation, as this is something we still want to avoid during normal driving.

Another issue that we came across when testing our first models was the control over throttle. Namely, when driving on different terrain, the throttle required to hold the same speed is very different. Reading this information out of the image is a lot harder than estimating the angle for steering. To solve this problem, one option is to try to use IMU (inertial measurement unit) data as one input to the network to help with the throttle control. Still, this is not the only variable that determines how fast the car drives. The speed is also highly dependant on the remaining battery voltage level.

The search for training data

By the end of the project, we aim to achieve a generalized model that is able to navigate as many different trails as possible. To achieve this, we require a versatile dataset. This includes varying the geometry of the track, as well as the amount of foliage around the track and the color of the soil. Lighting conditions can also have an effect on the results, so data in different weather conditions is required.

To build this versatile dataset, we collected data by driving the DonkeyCar in several different places in Tartu over the span of several days. To test how well the model generalized for different trails, we will train the model on some of the data and observe the loss on a validation set that was not a part of the training set. This will allow us to numerically represent the performance of the model on an unseen track.