My Experience with DeepRacer: A Headstart and My Journey
Deepracer🚗 is an AWS event that is conducted virtually and physically in 🏁 track. The main motive is to learn machine learning in specific reinforcement learning in a fun way where we get to enjoy the 😊 process.
Deepracer is a reinforcement learning technique to enable autonomous driving for the AWS DeepRacer vehicle.
Reinforcement Learning (RL) is the 🧠 science of decision-making. It is about learning the 💡 optimal behavior in an environment to obtain maximum 🏆 reward.
RL doesn’t need any 🏷️ labeled data, it takes a large amount of data, identifies patterns in the data, and makes decisions.
Terminologies you need to know for sure about RL:
💡 Agent — Our Deepracer vehicle is our 🚗 agent who is going to perform all our 🏎️ actions
🏎️ Environment — It is a place where the agent interacts, here it is our 🏁 race track.
🛣️ State — The position the agent is currently in refers to the state of the agent.
🚦 Action — The decisions that result in a movement made by our agent.
🔱 Policy — It is a set of actions that makes decisions for the next move. The policy gets updated after each move.
🏆 Reward — They are points awarded to our agent based on the decisions they make.
📈 Value — Evaluation of the actions made by the agent.
DeepRacer works based on the ⚖️ exploration-exploitation dilemma, which is a 🧠 fundamental concept in reinforcement learning. The dilemma is between choosing to 🔱 exploit the current best-known policy or exploring new policies to improve 🏆 performance.
→💡 Exploration is the process of trying new things and learning about the 🏎️ environment. This can be done by taking actions that the model has not taken 🔙 before or by exploring different parts of the 🏁 environment.
→🔱 Exploitation is the process of using the knowledge that the model has learned to make decisions that are likely to be 🏆 successful. This can be done by taking actions that have been successful in the 🔙 past or by taking actions that are predicted to be successful.
Diving into training your own 🚗 DeepRacer model, which is similar to 👩🍳 cooking in that it is a 💡 creative process that involves 👩🍳 trial and error, 🧪 experimentation, and a deep understanding of the 🧠 underlying principles.
Let’s 🥘 compare DeepRacer and cooking, and 👩🍳 cook our DeepRacer dish together.
The Deepracer Dish
🍳 First, you need your pan to start your dish which is your Deepracer console💻.
🥘 The primary ingredients required for our DeepRacer dish are:
→ Agent — 1 🚗
→️ Model — 1 🏎️
→ Track — 1 🏁
→ Hyperparameters as required ⚙️
👩🍳 Let’s start cooking, and here are a few steps that describe what you need to do, I’ll explain what they mean after we’re done cooking.
1️⃣ Step 1: Start your lab and enter into the deepracer 💻 console.
2️⃣ Step 2: Design your vehicle with the required cameras and sensors according to your physical 🚗 model.
Note: Your physical model won’t fit in with the wrong virtual model features
Step 3: Creating your model 🚗🏁.
You need to build a model and train it before racing on the track.
So to cook your deepracer curry base of the model you need to select the Track type 🏁, 🏎️ Race type 🏆, Algorithm 🤖 , Hyperparameters 🔀, Action space 🚦, Agent 🏎️ , Reward function 🏆 and Training period ⏳ .
Step 4: Let the model be cooked 🍳 i.e., training and we’ll taste it before serving ⏳🥘
🥘 Now let’s take a look at our curry base 🥘 ingredients,
🚗 Once you have entered your console, you can start building your DeepRacer model by choosing the ingredients that you need. These ingredients include:
- Track type: 🏁 The track type is the environment in which the 🚗 agent will learn to race. There is a wide range of 🏁 track types provided by the Deepracer platform and you can choose accordingly as per your 🥘 needs.
- Race type: 🏆 The race type determines the goal of the 🚗 training.
• Time trial🏁: The goal of a time trial is to get the 🥇 fastest lap time possible. This is a good way to learn the basics of racing and to get a feel for how the 💻 DeepRacer platform works.
• Object avoidance🏁: The goal of an object avoidance race is to avoid obstacles while driving as fast as possible. This is a more challenging race type, but it can help you improve your agent’s ability to make decisions in difficult situations.
• Head-to-head racing🏁: The goal of a head-to-head race is to race against other agents and finish 🥇 first. This is the most challenging race type, but it can be a lot of fun. - Training Algorithm: 🤖 The two training algorithms available in DeepRacer are Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC). Both algorithms work by learning a policy that maps states to actions. However, PPO is a 🔱 policy gradient algorithm, while SAC is a 🔱 value-based algorithm.
- Hyperparameters: ⚙️ The hyperparameters are the settings that control the training process. You can adjust these settings to improve the performance of your model.
- Action space: 🚦 The action space is the set of all possible actions that the agent can take. You can choose from a variety of action spaces, such as continuous or discrete.
- Reward function: 🏆 The reward function is the metric that the agent will use to evaluate its performance. You can choose from a variety of reward functions, such as the lap time or the number of obstacles avoided.
- Training period: ⏳ The training period is the amount of ⏱️ time that the agent will train. You can adjust this setting to improve the performance of your model
I guess that our model is well-cooked, let’s taste our freshly cooked model
Evaluating the model 🥘 consists of measuring the performance measure 🏆, which tells us how well the model is cooked 🥘 and I’ll be continuing this in the next part.
My Experience with DeepRacer
My journey with DeepRacer 🚗 was a lot of fun 🤩 and started through the Deepracer League conducted By KGiSL EDU and AWS virtual community races 🏁 and recently a workshop conducted by Thoughtworks, Coimbatore.
Tbh, It wasn’t super easy or fun in the beginning 😅, but eventually, it got better and turned out well in the end 👍. Let’s get to know what DeepRacer is, and I’ll share the mistakes I made and what I learned over time and training📚.
Deepracer 🚗 comes under the few things I wouldn’t regret trying 🤩.
At the start, I just went to the console 💻 and trained the model randomly by adjusting the hyperparameters ⚙️ and a few other requirements 📝, but I never touched the reward function 🏆. I know that’s not the right way to practice 😅, but I learned from my mistake 🥲 and grew 📈.
I learned a lot about reinforcement learning 🔱 and autonomous driving 🏎️ through this journey 🛣️. I also had a lot of fun racing against other models 🏁. If you’re interested in machine learning 🤖 or autonomous driving 🏎️, I highly recommend trying out DeepRacer 🚗.
Don’t be afraid to experiment. There is no one right way to train a DeepRacer model. The best way to learn is by trying different things.
Let’s connect socially:
➡️https://www.linkedin.com/in/akshayavarshieni/
