Parameters In Depth: DeepRacer Student League Guide

Images and foundational content are based on this official AWS DeepRacer Guide, but adapted and expanded for DeepRacer’s Student League found here. All code snippets and examples have been removed but can be found in the original documentation.

Welcome to my guide on the parameters provided for the DeepRacer Student League! Although AWS has done a good job with a base outline of these parameters, there are “hidden” elements and limits to many of them that can’t be found through the student console. Everything taken directly from the old guide is formatted like so:

DeepRacer is…

As well, throughout this guide I’ll be adding my thoughts on many of these parameters underneath, like so:
My thoughts: …

The Input Dictionary

Your reward function has one input: a dictionary usually referred to as params. It contains all the parameters in key-value pairs, where the key is always a string.

The params dictionary has the following key-value pairs:

{
    "all_wheels_on_track": Boolean,        # flag to indicate if the agent is on the track
    "x": float,                            # agent's x-coordinate in meters
    "y": float,                            # agent's y-coordinate in meters
    "closest_waypoints": [int, int],       # indices of the two nearest waypoints.
    "distance_from_center": float,         # distance in meters from the track center 
    "is_left_of_center": Boolean,          # Flag to indicate if the agent is on the left side to the track center or not. 
    "is_offtrack": Boolean,                # Boolean flag to indicate whether the agent has gone off track.
    "is_reversed": Boolean,                # flag to indicate if the agent is driving clockwise (True) or counter clockwise (False).
    "heading": float,                      # agent's yaw in degrees
    "progress": float,                     # percentage of track completed
    "speed": float,                        # agent's speed in meters per second (m/s)
    "steering_angle": float,               # agent's steering angle in degrees
    "steps": int,                          # number steps completed
    "track_length": float,                 # track length in meters.
    "track_width": float,                  # width of the track
    "waypoints": [(float, float), ]        # list of (x,y) as milestones along the track center
}

See below for more detailed descriptions of each parameter.

all_wheels_on_track

Type: Boolean

Range: (True:False)

A Boolean flag to indicate whether the agent is on-track or off-track. It's off-track (False) if any of its wheels are outside of the track borders. It's on-track (True) if all of the wheels are inside the two track borders. The following illustration shows that the agent is on-track.

If all wheels are on track, this parameter is true

The following illustration shows that the agent is off-track.

If one or more wheels are off track, this parameter is false

My thoughts: I like this parameter a lot, and almost all reward functions I make use of this parameter. I prefer it a lot more than the “is_offtrack” parameter because I feel that once the agent is already off track, it’s too late to course correct. I think that “all_wheels_on_track” gives the agent a lot more reaction time to react to going off track.

closest_waypoints

Type: [int, int]

Range: [(0:Max-1),(1:Max-1)]

The zero-based indices of the two neighboring waypoints closest to the agent's current position of (x, y). The distance is measured by the Euclidean distance from the center of the agent. The first element refers to the closest waypoint behind the agent and the second element refers the closest waypoint in front of the agent. Max is the length of the waypoints list. In the illustration shown in waypoints, the closest_waypoints would be [16, 17].​

distance_from_center

Type: float

Range: 0:~track_width/2

Displacement, in meters, between the agent center and the track center. The observable maximum displacement occurs when any of the agent’s wheels are outside a track border and, depending on the width of the track border, can be slightly smaller or larger than half the track_width.

My thoughts: This parameter is usually used in conjunction with track_width to normalize how far from the center a user agent, is by doing “1-[distance_from_center / (track_width/2)]”. This formula will return 0 when the car is in the center of the track, and 1 when on the edge of the track.

heading

Type: float

Range: -180:+180

Heading direction, in degrees, of the agent with respect to the x-axis of the coordinate system.

My thoughts: This parameter is usually used in conjunction with “closest_waypoints” and “waypoints” to see if the car is pointing in between the next 2 waypoints.

is_left_of_center

Type: Boolean

Range: [True : False]

A Boolean flag to indicate if the agent is on the left side to the track center (True) or on the right side (False).

My thoughts: Some people have done cool things with this parameter, such as using (x, y) coords or waypoints to determine if the car should be left or right of center. It seems to be a midpoint between having a more hands-off rewards model and having it follow a racing line, and I’ve never really used it.

is_offtrack

Type: Boolean

Range: (True:False)

A Boolean flag to indicate whether the agent has off track (True) or not (False) as a termination status.

My thoughts: I don’t like this parameter. Refer to the “all_wheels_on_track” parameter.

is_reversed

Type: Boolean

Range: [True:False]

A Boolean flag to indicate if the agent is driving on clock-wise (True) or counter clock-wise (False).

It’s used when you enable direction change for each episode.

My thoughts: This parameter is not referring to whether the speed on your car is negative, which is something I didn’t realize for a while. It’s supposed to be so your model can use a different function dependent on direction, which in the context of a simulated model that goes one way feels a bit useless.

progress

Type: float

Range: 0:100

Percentage of track completed.

My thoughts: This is possibly one of the most important parameters when combined with steps, maybe just right behind speed. When dividing progress by steps (ie progress/steps), it gives a way to measure how fast your model is going around the track while still allowing your model to slow down around corners if needed (as opposed to directly tying the reward to speed).

speed

Type: float

Range: 0.0:5.0

The observed speed of the agent, in meters per second (m/s).

In the DeepRacer Student Console action space, the speed is limited to 1 m/s, as opposed to what the parameters specified.

My thoughts: Speed is important. Care about speed.

steering_angle

Type: float

Range: -30:30

Steering angle, in degrees, of the front wheels from the center line of the agent. The negative sign (-) means steering to the right and the positive (+) sign means steering to the left. The agent center line is not necessarily parallel with the track center line as is shown in the following illustration.

My thoughts: This parameter is like the “left_of_center” parameter, in the sense that I’ve seen cool stuff happen with it, but it is hard to use. Like “left_of_center”, it’s usually used in conjunction with x y coords or waypoints to encourage steering around corners (a higher steering angle) or going straight during long stretches of the track (a lower steering angle).

steps

Type: int

Range: 0:Nstep

Number of steps completed. A step corresponds to an action taken by the agent following the current policy.

The “steps” parameter is incremented by 1 every time that your reward function (and all the surrounding, auxiliary code) is run. This happens about every 1/15 seconds, meaning 1 second ≈ 15 ± 1 step. Further, steps are reset to 0 every lap.

My thoughts: This is possibly one of the most important parameters when combined with progress. When dividing progress by steps (progress/steps), you get a measure of how much progress you make per step, which is fundamentally the thing you want to increase the most in your model.

track_length

Type: float

Range: [0:Lmax]

The track length in meters. Lmax is track-dependent.

track_width

Type: float

Range: 0:Dtrack

Track width in meters.

My thoughts: This parameter is usually used in conjunction with “distance_from_center” to normalize how far from the center a user agent, is by doing “1-[distance_from_center / (track_width/2)]”. This formula will return 0 when the car is in the center of the track, and 1 when on the edge of the track.

x, y

Type: float

Range: 0:N

Location, in meters, of the agent center along the x and y axes, of the simulated environment containing the track. The origin is at the lower-left corner of the simulated environment.

My thoughts: These parameters seem to be obsolete, as waypoints are an easier way to track where your car is. They could be useful, but I have never seen them used.

waypoints

Type: list of [float, float]

Range: [[xw,0,yw,0] … [xw,Max-1, yw,Max-1]]

An ordered list of track-dependent Max milestones along the track center. Each milestone is described by a coordinate of (xw,i, yw,i). For a looped track, the first and last waypoints are the same. For a straight or other non-looped track, the first and last waypoints are different.

My thoughts: In the profession DeepRacer (ie. the non-student league), this is probably the most important parameter combined with speed; in the pro-league models are hyper fit to the track, and waypoints are a way to be able to fit a model to the precalculated racing line. They are less of a META in the student league, although still highly used, but should still be understood before discounting them.

Conclusion

Most of the DeepRacer parameters are pretty straightforward, which is nice and easy for beginners to get started with. However, some of them can be frustrating to understand without help (looking at you, “steps”).

While a lot of the parameters are simple, the skill ceiling for utilizing these parameters is high and takes a combination of skill and testing to use well, which I hope to cover in further articles.