In my last article on this project, I talked about building and setting up the Donkey Car using the donkey library. Now that it can drive itself, what’s next? There is still a bit of distance between how I drive vs the model. The current model is only good enough to go around my track in low speed, and even in this low speed it still crashes quite often. The official blog does have some very good suggestions, but I really want to gain a better understanding of how the current system falls shorts. In order to do that I decided to take a more data-oriented approach. Below are the things I did to understand the data better and to improve result without actually changing the model.
Exploratory data analysis
Here I used Jupyter notebook to look at data using Pandas and Matplotlib. If you are not familiar with Jupyter notebook, look here. First I wanted to see how balanced my data is. I was concerned that because my track is fairly small due to the space limitation in my living room, my steering would tend to go to the extremes due to sharp turns. Plus at the time I was still using my phone as control, it was hard to provide precise steering with my finger sliding on a screen. I was worried that training on unbalanced data can result in the model making only extreme predictions which can prevent the car from staying on track well.
It turns out, the data is not as bad as I thought. The steering angle range from -1 to 1 with -1 being all the way left and 1 being all the way right. Although there are clear peaks on the extremes and the center, they don’t seem too out of proportion.
Another graph that I found helpful to look at was the plot of the steering angle against the camera frames. It provides some helpful information about how smoothly the car is driving (in this case manually).
Seems like the data is not smooth at all. The above example shows a very bumpy curve probably due to my awkward attempt at making a tight turn with my finger sliding across the screen.
Getting better data
As I mentioned in previous piece on this project, I made a modification to add support for using the original RC controller with an Arduino for better control. Based on the brief data analysis above I wanted to find out if getting smoother control data this way can help with model prediction.
You can also achieve better control by buying a Bluetooth joystick controller and connect it with the Raspberry Pi following these steps, which is the fastest way to achieve that. I decided to go through the trouble mainly to force myself to learn more about the donkey car control library, and also because I had an Arduino lying around and didn’t want to buy an extra controller while abandoning a perfectly good one.
If you are interested in going this route or just want to learn more about Arduino, here is how I did it with resources below. Otherwise feel free to skip to the next paragraph. From reading the donkey library I learned that I could modify the web controller to fit my use. Instead of receiving input from the web interface, it would listen to the serial(USB) port where the Arduino is plugged into. The important parts are to make sure that Arduino is giving the correct signal and that the Raspberry Pi is receiving it correctly. On the Arduino side, I used the PulseIn function to measure PWM signal from the car receiver and to turn it into usable signal to be sent to the Pi. (PulseIn is not the most efficient way to use Arduino’s CPU, but since it’s not doing anything else, this is good enough for the time being). Here are the codes:
- Arduino sketch to read PWM signal and output between 0 and 200 in Serial Port
- Python SerialController code to replace the web controller on Raspberry Pi. You can also combine it with the original web controller code to make it to work with existing interface. This is in my future todo list.
- You will also have to change manage.py a little to replace the web controller part with the new SerialController part:
from donkeycar.parts.serial_controller import SerialControllerctr = SerialController()
V.add(ctr, outputs=['user/angle', 'user/throttle', 'user/mode', 'recording'], threaded=True)
So how does data differ between the original control and the new one?
Here are two plots on the same turn. Seems that with the new RC serial controller module, I was able to achieve much smoother turns.
Noise Filtering
Before jumping into working on the model, I wanted to explore a little about simple filtering. Since the car still crashed quite often during auto pilot, I decided to look more closely at the pilot driving data. By default, the donkey library only auto record manual driving data for training purpose. Fortunately it is quite easy to change that to record pilot action during autonomous mode. Just add the “pilot/angle” keyword to the tub’s input array in the main drive loop file and record manually by pressing the record button on the web interface.
After obtaining pilot driving data, I looked through all images in bulk to find the few frames just before a crash and looked closely at the angles predicted at those frames. I found that a lot of the crashes were caused by one extreme outlier prediction made by the pilot. For example, if the car is making a left turn, one wrong prediction to steer all the way to the right would cause it to go off track.
Based on that info, I added a simple filter to discard outlier prediction that deviates more than 25% from the last one. After some experimentation, I also added a constrain to this filter to only turned on every few frames. The constrain was needed because once an outlier prediction was detected, its data was discarded; and the filter could only rely on the last known good prediction to compare with what comes next. Predictions made by each frame, however, could often deviate more than 25% from two frames ago in normal condition, which could cause the filter to overcompensate. With constrain added, the resulting filter was good at only discarding outlier once in a while, which wasn’t a perfect solution, but it really helped the car stay on track for much longer.
This method seems to work for a period of time. Although it is not reliable for the long run, it demonstrates what simple filters can do to compensate prediction failures by putting constraint on driving behavior. Next, I want to explore how to actually improve the model so it makes better predictions in the first place. Go to my next article to read about:
- Setting up an environment for deep learning
- Finding a better model architecture
- Testing and comparing models