How to start with AWS DeepRacer🚀: My Experience at the Thoughtworks Workshop👾

What’s up, racers? Let’s get this started!🙌

I’m here to drop some knowledge I got from the AWS DeepRacer workshop that went down in Thoughtworks, Coimbatore💡.

I’ve been to the 48-hour AWS DeepRacer event conducted by KGiSL Edu, back in 2022 🕒.

But a week before(Aug 31st), I went to ThoughtWorks with a few of my KGX colleagues, all hyped up to learn new things (or maybe just to relearn the ones I already screwed up)🤓.

Before we get started with Deep Racer, let me tell you what it is for all you newbies out there.🚗💨

“AWS DeepRacer is the first self-driving toy car specifically designed to help developers get their hands dirty with reinforcement learning.”🤖

What is Reinforcement learning?🤔

“Reinforcement learning is one of the machine learning techniques where an agent learns to behave in an environment by trial and error, without any explicit instructions🧠.”

Reinforcement learning is like training a dog 🐶 to follow your instructions, except the dog is a computer program🖥 and the treats are virtual. The dog learns by trial and error and is rewarded for taking actions that lead to good outcomes🍪. If the dog does something wrong, it doesn’t get a treat🫣, and it’s more likely to try something different next time👾.

DeepRacer is a reinforcement learning (RL) platform that enables you to train autonomous vehicles to race on a virtual track🎮. It uses a simulated environment to train your model, so you can experiment and iterate quickly without having to worry about damaging a physical vehicle🚘.

Double bammm!!💥💥

Before we can train our DeepRacer car, we need to learn the lingo.

Here are the terms you need to know:📚🤓

• Agent: The thing that’s doing the learning. In our case, it’s the car.🚗
• Environment: The world that the agent is interacting with. In our case, it’s the racetrack.🏁
• State: The current condition of the environment. In our case, it’s the car’s position, speed, and angle.🚦
• Action: Something that the agent can do. In this case, it’s steering left, steering right, or accelerating.↖️↘️
• Policy: A rule that tells the agent what action to take in each state. In our case, it’s a neural network that we’re trying to train. 🧠🤖
• Reward: Something that the agent gets for taking a good action. In our case, it’s a decrease in lap time.⏱️🏆
• Value: How good a state is. In our case, it’s how close the car is to the finish line.🏁🏎️

How to Train Your First AWS DeepRacer Car!!

First, let’s get into the AWS DeepRacer console.🔍

Once you log in to the AWS DeepRacer console, you will see the “Your models” page under the “Reinforcement learning” heading. From there, you can create a new model by clicking the “Create model” button🚀.

After clicking the Create model button, you will be taken to the Model details page📝

On this page, you will need to enter the training details for your model, such as the model name and description📋.

You will also need to select the simulated environment in which you want to train your model🎮.

So, you’ve decided to race your car💪.

But what kind of race do you want to run? 🤔

There are three options:

• Time trial⏱️: This is the classic race against the clock. You’ll need to see how fast you can complete the course.
• Object avoidance🚧🙅‍♂️: This is a bit more challenging. You’ll need to avoid obstacles on the course while still trying to finish as fast as possible.
• Head-to-head racing🤜🤛: This is the most competitive race. You’ll need to beat your opponent to the finish line.

Once you’ve chosen your race type, you need to choose your training algorithm. There are two options:

• Proximal Policy Optimization (PPO)🤖: This is like a single-minded coach. It focuses on getting you to the finish line as fast as possible, no matter what 💪.
• Soft Actor-Critic (SAC): This is like a more laid-back coach. It’s happy to let you explore the course and try different things, as long as you eventually make it to the finish line🚗🏁.

So, which race type and training algorithm should you choose? 🤷‍♀️

If you’re competitive and want to win, go for head-to-head racing with PPO🏆. If you’re more laid-back and want to explore the course, go for object avoidance with SAC🏆.

The hyperparameters are like the ingredients in a recipe🥣. They control how the training process works👨‍🍳, and they can make a big difference in the performance of the model🌮.

• Gradient descent batch size: This is the number of experiences that the model tastes at once 😋. Higher batch size can make the model more likely to choke.
• Epochs: This is the number of times the model eats the entire recipe 🍽️.
• Learning rate: This is how much the model adjusts its recipe after each bite 📈.
• Entropy: This controls how much the model likes to try new things. A higher entropy means that the model is more likely to try new things, while a lower entropy means that the model is more likely to stick with what it knows🤔.
• Discount factor: This controls how much the model cares about future rewards. A higher discount factor → cares more about future rewards, while a lower discount factor → cares more about immediate rewards.⏳
• Experiences: This is the number of times the model goes off track before it starts over. A higher number of experiences means that the model can learn from its mistakes, but it can also make the training process take longer.🚗

Next, the action space is where you get to decide how your car will behave. It’s like a menu, with all the different things your car can do😍. There are two main options on the menu:

• Continuous action space- It is like a buffet🍨. You can choose any steering angle and any speed you want. This is a lot of freedom, but it can also be a lot of work. You have to try a lot of different combinations to find the best ones🥂.
• Discrete action space- It is like a fast food restaurant🍕. You can only choose from a few pre-defined options. This is less freedom, but it’s also less work. You don’t have to try as many combinations to find the best ones☕.

The steering angle is how much you turn the wheels🔄. The speed granularity is how many different speeds you can choose from📃.

If you have a lot of time, you can go with continuous action space⏰. If you’re in a hurry, you can go with discrete action space🏃‍♀️.

But no matter which action space you choose, be careful what you order👨‍🔬. Once you’ve defined your action space, you can’t change it. So make sure you choose wisely!🧠

Now you need to choose your vehicle🚗🤔.

There are a few different vehicles to choose from, each with its own strengths and weaknesses. You can choose the vehicle that best matches your physical model, or you can create your own vehicle with the required specifications💪📋.

Once you’ve chosen your vehicle, you need to give it a reward function🎁. This is what tells the vehicle what it should do to get rewarded. The reward function can be customized, or you can use one of the pre-defined reward functions.🤌

The training time depends on the complexity of the track and the difficulty of the race type. It can take anywhere from 5 minutes to 1440 minutes⏱️🏁.

During training, you can watch a live simulation of the vehicle as it learns to race. You can also see a reward graph📈, which shows how the vehicle’s performance is improving over time 🏎️.

Once the training is complete, you can evaluate the vehicle on the required track. This will give you an idea of how well the vehicle performs in a real race🏁🚗.

Hooray🏁🚗! We’ve done it! We’ve trained a model!🤖

Once you’re happy with your model🤩, it’s time to tune the hyperparameters. This is like fine-tuning your recipe😋. You can experiment with different values to see how they affect the performance of your model🧪.

Once you’ve found the best hyperparameters🫡, you can participate in the Racing League 🏁. This is where you can race your model against other models from around the world🌍.

The Racing League is a great way to test your model and see how it performs against other models🏆📚. It’s also a great way to learn from other people and improve your own skills.

So, what are you waiting for? Start tuning your hyperparameters and get ready to race!🏇

Finally, I’d like to thank Navaneeth Malingan, KGX for sending us to Thoughtworks, even though I’m pretty sure they only did it because they were tired of us being around the office all the time😄🙌.

Feel free to Connect🎯

➡️https://www.linkedin.com/in/elakiya-sekar-28465b220/

➡️https://www.instagram.com/elakiya__sekar/

➡️meelakiya24@gmail.com