The left/right skew is due to driving the car around the track in one direction only and can be eliminated by flipping each recorded image and its corresponding steering angle. More troublesome is the bias to driving straight: the rare cases, when a large steering angle recorded are also the most important ones if the car is to stay on the road. One possible solution would be to let the car drift to the edge of the road and recover before a crash occurs. I tried this, but found it an unsatisfactory solution, because in that case the car still goes straight most of the time — direction “off the track” — with a few large steering angles sprinkled on top. As a result a CNN trained on such data typically does not even complete the training track, unless the training data is ‘just right’. I got a CNN to drive the car around the training track this way, but the model failed on the test track.
Long Answer: It is possible and I am actually surprised developers haven’t given it the attention it deserves. As far as
scikit-learn is concerned, the JS people have made their own set of libraries to counter it, and I am gonna use one too. But first, a little bit about Machine Learning. Feel free to board this rocket 🚀 and jump to the code, though.