Creating an EC2 cron job for automatic submitting of an AWS DeepRacer model to the AWS DeepRacer League

AWS DeepRacer https://aws.amazon.com/deepracer/ is an AWS-managed service for studying the basics of Reinforcement Learning (one of the Machine Learning types) in a gamification mode.

This post is for developers who are already acquainted with the AWS DeepRacer service and its main capabilities and want to improve the ranking of their models in the AWS DeepRacer League.

If you have looked into the statistics of AWS DeepRacer league races, you might have noticed that some participants have a huge number of submission attempts. The picture below shows the number of submissions for the top-10 of one of the Pro leagues.

A large number of submission attempts is one of the important factors in achieving a higher ranking in any DeepRacer race. You may know that each evaluation attempt for your ML model gives slightly different results. The difference between two evaluation attempts in a race for the same model may be around 30 seconds. This is huge because all the race participants are ranked by time with a difference of milliseconds, and 30 seconds less time brings you much higher in the ranking.

Therefore, if you know that your model can perform better than the race evaluation, it is very important to make more submission attempts to try your model to show its best. For sure, you can do it manually, but it requires action from your side, and the total number of attempts may not be enough to show the best timing by your model.

How can it be improved? With automation!

AWS DeepRacer community driven by DeepRacer fans has developed a set of tools/utilities for improving the performance of DeepRacer models. All the projects are available on GitHub: https://github.com/aws-deepracer-community. Among the projects, there is a “deepracer-utils” Python library that contains helper classes for DeepRacer model performance analysis. And more importantly, it contains an enhancement for AWS CLI that makes DeepRacer commands available in the command line (currently DeepRacer is absent in the official AWS CLI).

Let’s check how to install and use this enhancement.

For the automatic model submission, my setup is an EC2 instance with a submission job scheduled by Linux crontab.

I have tried a Lambda function and unfortunately, it does not work, because it requires the CLI extension that should be packaged as a Lambda layer. But the size of this layer exceeds the maximum limit of packaged Lambda layer size.

The step-by-step guide to the setup is given below.

Part 1: Create an EC2 instance role

1. Go to the IAM service in the AWS console.
2. Select the “Policies” left menu item and click the “Create Policy” button in the top right.

3. Select the “JSON” tab and fill in the JSON policy given below.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "deepracer:ListLeaderboards",
                "deepracer:GetModel",
                "deepracer:ListModels",
                "deepracer:GetLeaderboard",
                "deepracer:CreateLeaderboardSubmission"
            ],
            "Resource": "*"
        }
    ]
}

4. Click “Next: Tags” button, add tags if you want to, and click “Next: Review”.

5. Provide a name for the policy (e.g., “deepracer-model-submission-policy”) and click “Create Policy”.

6. Click on the “Roles” left menu item and click the “Create role” button in the top right corner.

7. Select the “AWS Service” option and the “EC2” use case and click “Next”.

8. Select the policy that was just created and click “Next”.

9. Specify the role name (e.g., “ec2-deepracer-submission-role”) and click “Create role” in the bottom right corner.

Part 2: Launch an EC2 instance

1. Go to the EC2 service in the AWS console and click the “Launch instances” button in the top right corner.

2. Give a name to the instance, e.g., “deepracer-sumbission”.

3. Leave the AMI and the instance type as set by default (Amazon Linux 2 x86 and t2.micro)

4. Select an existing key pair (or create a new one).

5. Leave other settings by default and scroll down to the “Advanced details” section.

6. Expand the “Advanced details” section. In the “IAM Instance Profile” field, select the IAM role that we created in the previous part in the “IAM Instance Profile” field.

7. Find the “User data” input and insert there a script that will upgrade AWS CLI there:

#!/bin/bash
# Uninstall AWS CLI version 1
yum update -y
pip3 install --upgrade --user awscli
pip3 uninstall awscli -y
# Install AWS CLI version 2
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install -i /usr/local/aws-cli -b /usr/local/bin
source ~/.bash_profile

(Just to recap: user data scripts are executed when an EC2 instance is provisioned with the “sudo” permission).

8. Wait some time until the instance is running with 2/2 status checks passed.

Part 3: Install the DeepRacer CLI extension and configure the model submission job

1. Connect to the instance by SSH (just go to the instance, click the “Connect” button in the top right and select an option that you prefer for connecting to your instances).

2. Run the following commands one-by-one:

pip3 install deepracer-utils
python3 -m deepracer install-cli --force

Once the execution is completed, try the following command to validate that the DeepRacer CLI commands are available:

aws deepracer help

If the output contains information about the DeepRacer CLI commands, the installation was successful.

3. Find the ARN of the model that you want to submit to a race by the following command:

aws deepracer list-models --model-type "REINFORCEMENT_LEARNING"

The output will contain a JSON with models available in the AWS DeepRacer service.

Note an ARN (“ModelArn”) of the model that you will submit to a race.

4. Go to a race of the AWS DeepRacer league to which you want to submit the selected model.

5. Copy a part of the page URL after league/ which is the encoded ARN of the race leaderboard.

6. Go to any online URL encode/decoder service (e.g., https://www.urldecoder.org/), paste the copied URL part, and click the “DECODE” button.

Note the decoded ARN of the race.

7. Go back to the SSH session with the EC2 instance and try to submit the model to the race by the command:

aws deepracer create-leaderboard-submission --model-arn "${MODEL_ARN}" --leaderboard-arn %{LEADERBOARD_ARN} --region us-east-1

If the command was successful, you will see nothing in the output, but the model submission will appear in the race in the AWS DeepRacer console:

8. Create a bash script with the command.

Create a bash script with any name (e.g., “submit.sh”) and put the command for submission there. Save the file.

9. Create a crontab job by the command:

crontab -e

Edit the file and specify the cron for the model submission frequency and command to run the script by this cron. E.g., the following setup will run the script with submission every 10 minutes:

*/10 * * * * sh /home/ec2-user/submit.sh

Use any online cron generator (e.g., https://crontab.guru/) to create the cron expression for your custom schedule.

If you don’t see the model submission in the AWS DeepRacer console, run on the EC2 instance crontab -l command and it will show if you have any issues with the cron execution.

Conclusion

In this post, we learned how to establish an automatic submitting of an AWS DeepRacer model to a virtual race using the cron expression from an AWS EC2 instance.