Expedia Group Technology — Data
Racing Cars in the Brisbane Office
Using Reinforcement Learning to race an autonomous machine around a physical track
As a part of Expedia Group’s partnership with AWS we recently took an amazing opportunity to host a DeepRacer competition in our Brisbane office. DeepRacer is designed to introduce people of all backgrounds to Machine Learning. The goal of the competition is to engineer a control loop for an autonomous toy racing car that enables the car to complete a full circuit of a physical race track in the shortest amount of time. This control loop is constructed using a Machine Learning technique called Reinforcement Learning.
Reinforcement Learning encourages an autonomous machine to perform certain actions. Using the AWS tools, it is possible to replay recorded race tracks to the algorithm while rewarding the racer for successful decisions and penalising failures. For example, if the racer detects a leftwards turn in the track, the correct action to take is to “turn left” and this is rewarded while “turn right” is penalised. If you are a parent with young children, this process will feel very familiar. After a suitable amount of parenting (formally called model training), the racer can be put on a new track and in theory is autonomous enough in decision making to follow the circuit. But, to the chagrin of the participants and entertainment of the crowd, things don’t always go as planned.
The DeepRacer cars employ an AWS DeepLens camera mounted on the front of the vehicle to record a video at 15 frames per second. Every frame of this video stream captured by the DeepLens camera is downsampled and greyscaled, then input into Reinforcement Learning model created by each team. In addition to the training described above, teams were able to tweak their individual models in various ways. For example, the maximum speed of a DeepRacer car is approximately 60km/hr (~40miles/hr). At this speed, the racer can travel nearly two meters in the time the DeepLens camera can capture a single video frame. Racing like this is very exciting but short lived as the DeepRacer hurtles itself straight into the nearest wall. Therefore, one of the key parameters to tweak is the maximum power of the DeepRacer. Too much, and the racer crashes out. Too little, and you finish far down the podium. Other parameters included using waypoints or not, training on multi-directional tracks or not and deciding which track signals to use as a target (ie: middle of the track or the left border).
We worked with our AWS Technical Account Manager to organise this event and frankly, without his help the event could not have gone ahead. Thank you Graham! We had to commandeer our office lunch room for a day to set up the race track and everybody, racers and pit crew, went through some rigorous training (basically, we gave them passwords and asked them not to run bitcoin mining operations on the AWS accounts we set up for the event).
The hard work put in behind the scenes allowed us to bring together a unique mixture of Expedia Group™️ staff in a fun and engaging event. For example, we brought together both engineering and non-engineering staff. From the non-engineering staff, I especially want to do a big shout out to the Wotif.com eCommerce team for the motivation and energy they showed in getting behind this event. Their aptly named team (“Wot the Truck are we doing?”) ended up taking fifth place and I think that deserves a photo spot.
Likewise, this event was a good opportunity to bring together engineers from across the Expedia Group. We had two teams (and some wonderful pit crew) from SilverRail Technologies participate and we also had a podium ranking team from our Sydney office. On that note, congratulations to Alok & Jerry (collectively, and less aptly named, “The Team”) for taking out first place and the other trophy winners too. The first place circuit time was clocked at 13.2s which is 1s behind the Expedia global best.
The feedback from the event has been overwhelmingly positive with several people asking for a chance to compete again. It has clearly given us all a chance to learn about Machine Learning but also it has brought us together as an extended group. Thank you everybody who was involved for making the event such a pleasure to coordinate.
