An Introduction to AWS DeepRacer from a 2020 World Championship Finalist
DBS’ top competitor shares his tips on how you can get started on your AI/ML learning journey.
In 2020, I received an email to DBS staff about a new initiative the Bank had joined — the AWS DeepRacer League. Staff was invited to teach a 1/18th scale racing car’s software through machine learning and compete for the fastest time around a track. As a data enthusiast and a member of our Customer Management & Analytics team, my interest was piqued.
Little did I know that my passion for artificial intelligence (AI) and machine learning (ML) would eventually lead me to set an unofficial world record time and take me to AWS’ 2020 world championships. Now, I’d like to share with you some tips to get started on your own AWS DeepRacer journey.
In less than an hour — if you have a basic understanding of the codes and methodologies — you can learn how to prepare your model for training. All you need is a passion for AI and ML.
1. What is AWS DeepRacer?
Organised by Amazon Web Services Inc., the AWS DeepRacer competition challenges AI/ML enthusiasts to train and programme an autonomous 1/18th scale racing car. Since its launch, it has also become a popular space for organisations to compete against one another. In 2020, DBS was one of the corporate organisations. Through competing in regional events, the year’s top racers are invited to AWS’s re:Invent Championship. In 2020 I was fortunate enough to be invited to this event and represent both DBS Bank (Hong Kong) and Hong Kong.
2. Getting Started
A great place to embark on your DeepRacer journey is to watch AWS’s 30-minute training video on YouTube. Watching the video will give you a clearer sense of what DeepRacer is about and — what’s more — you’ll have already created a model as instructed throughout the video.
The AWS DeepRacer competition features three types of races:
1. Time Trials
2. Obstacle Races
3. Head-to-Head Races
This guide will focus on how to excel at time trials. Here are some of my top tips, learned from my AWS DeepRacer experience.
A. Understand the technical boundaries of the DeepRacer’s reward functions
The input parameter page is a valuable resource throughout your DeepRacer journey. It’s important to familiarise yourself with the parameters before coding your reward function — after all you cannot design the right function without knowing the DeepRacer’s limits.
B. Understand the track you’re racing
Familiarity with the course will give you a competitive advantage. For example, you can examine the track and identify when your model should be slowing down for an upcoming turn or when it should be accelerating on a straightaway. Other parameters to consider include the track length, the track’s width, as well as the track starting point. These variables may affect your reward function design and the log analysis — a useful tool for optimizing your model. You can gather all this information on the DeepRacer Console.
C. Before you code, imagine the model’s performance and reaction to your design
Going back to the practice model you created during the 30-minute video, the only reward function was to move the car along the central line of the track. This, however, will not be the fastest way to get around the track. Before you design your model, imagine how you want your car to traverse the track. Knowing this will help you design a better reward function.
3. Designing Your Reward Function
I highly recommend reading Falk Tandetzky’s article on how to design and code your model’s reward functions. While the goal of any reward function in a time trial is to have a shorter lap time, the key questions are ‘how?’ and ‘when?’ For example, what is the optimal racing line? Where should the model look go to full speed or cut a corner? The following section will highlight the two approaches to designing reward functions.
Fully Autonomous
A fully autonomous model means you incentivize the reward function to learn and act by simply defining a certain goal (e.g. shorter lap time). Through reinforcement, the model will gradually learn how to achieve the defined goal. As the model is learning independently of human input, this approach is more costly — with the possibility of the model’s training taking hundreds of hours and requiring high amounts of computation power. While this approach may be the preference of experienced AWS DeepRacer participants and AI/ML experts, I would recommend the following semi-autonomous approach.
Semi-Autonomous
The majority of AWS DeepRacer participants opt for a semi-autonomous approach when designing their reward function. A semi-autonomous model means you can specify certain add-ons, conditions and behaviors that help to incentivize your model. This in turn helps speed up the training time — reducing cost and computation power.
By default, your model is coded to direct itself back to the course’s centerline. In other words, the model will race as fast as it can if you reward speed. The details of using the optimal line and the how-to will not be covered in this guide. To learn more about racing lines, I would recommend referring to the github link here — created by the AWS DeepRacer cdthompson. This is a popular resource for many DeepRacer enthusiasts.
4. How to Train Your Model
There are three training approaches that feature various trade-offs:
Console Training
Having done the 30-minute training video, you will already be familiar with this training approach. This is the best starting point for newcomers as the infrastructure is managed by AWS and you can focus on your model. There is also a standard charge per hour, so you can easily budget your overall training cost.
Local Training
You can also leverage your own computer’s capacity to train your model. The benefits of this approach are higher training efficiency and a more highly controlled training environment (e.g. console training cannot account for variables such as lighting). However, to set up local training on your own computer, it requires prior knowledge of Linux. If you are familiar with Linux, you can follow the github guide here. This capability was actually set up by fellow members of the AWS DeepRacer community — credit to Crr0004, AlexSchultz, Richardfan1126 and LarsLL and others. Another trade-off of local training is that all the training and infrastructure are managed by you. An important tip is to therefore remember to save your trained models.
Cloud Training
You may now be asking if it is possible to leverage Cloud rather than a computer’s capacity for local training. The answer is yes — there are now repositories supporting Azure and AWS cloud training for AWS DeepRacer (credit to LarsLL). While you are not required to take care of the infrastructure with this training approach, it is still important to take good care of your models.
5. Next Steps
Now that you know how to design your reward function and train your first model, it’s time to train your second model. The challenges for you will be:
1. Optimising your training by tuning the hyperparameters
2. Learning from your failures/mistakes by using log analysis
Full details can be read on the RayG blog on AWS, big shoutout to RayG, our DBS DeepRacer leader
3. Enhance your reward function(s) by balancing speed and stability
4. Deploy your model into AWS virtual races or even a physical one
5. Reiterate the above steps just like what a machine learning model would do until you earn yourself a ticket to the re:Invent Championship!
You’ve now embarked on your AWS DeepRacer journey. The next step is to deploy your model to a 1/18th scale DeepRacer car and to race it on a physical track. There is much to learn, especially on the topic Sim2Real (Simulation to Reality) Challenge. Machine Learning is an exciting topic to study, and it’s practically impossible to know everything there is to know about it!
I hope you found this walkthrough guide to be a useful resource and are inspired to kickstart your own AWS DeepRacer journey. From my experience in the AWS DeepRacer initiative, I have had the opportunity to network not only with the Bank’s experts on AI and ML, but also those across Asia and the rest of the world. DBS Bank’s AI/ML community continues to grow, and I’ve been grateful to be able to exchange ideas with the team during the championship. It’s also been inspiring to see how there were AWS DeepRacer enthusiasts from across the Bank’s units — we had tech guys who supported our backend systems, as well as data analysts working with the business.
Last but not least, I’m grateful for the support this initiative has received from the very top echelons of the Bank. To have leaders such as Piyush Gupta (our Group CEO), Sebastian Paredes (our Hong Kong CEO) and Alfian Sharifuddin (Head of Technology & Operations for Hong Kong and China) provide encouragement and financial support has been incredible and a big inspiration to us all.
About the Author
Kevin Suen is a Hong Kong-based data enthusiast and one of the global top 100 AWS DeepRacers. He joined DBS Bank in 2018 as DBS HK Technology & Operations Graduate Associate. In 2020, he represented both DBS Bank (Hong Kong) and Hong Kong at the AWS DeepRacer re:Invent Championships.
