Racing towards Intelligent Automation
This post is jointly written with Declan Robinson.
Working closely with our clients, we occasionally have the opportunity to do something a little bit outside of the norm. In this post, we (Declan and Arjen) tell the story of an instance where we were lucky enough to help out Belong with a fun learning event.
Why DeepRacer?
As a bit of background, Belong is in the process of setting up an Intelligent Automation (IA) Guild. Intelligent automation can be seen as an application of machine learning (ML) or artificial intelligence (AI) in the space of more traditional process automation, to both improve efficiency as well as reduce the human effort required for less interesting, menial tasks. The IA Guild is therefore seen as a key enabler of Belong’s strategy, and as such is vital in terms of how they attract and retain key talent whilst fostering and growing skills in this area across the company.
To get people interested in the guild, Belong wanted to do something fun for the kick-off that is still related to AI/ML. As it happens, there is a perfect tool that embodies both machine learning and having fun: AWS DeepRacer. And so was born the Belong DeepRacer event.
DeepRacer was introduced by AWS at their re:Invent conference last year as a way to learn about reinforcement learning. This is done by letting you train self-driving racing cars. Let me repeat that, DeepRacer lets you train self-driving cars.
Reinforcement learning itself is also a concept that is easy to understand, as it’s a way of training a model that comes natural to most of us. All you need to do is reward your car when it behaves correctly. For example, if the car stays on the track it gets a reward (unfortunately, this is simply a number but nothing stops you from pretending it’s a cookie). Ideally you’d want to design your model such that the car will get the most cookies for completing the track in the shortest time.
The hands-on nature of AWS Deepracer makes it a great tool for breaking down the barriers of entry into a pretty daunting topic, so for our team at Belong it made sense to host our own racing event to try to drum up some excitement and get the ball rolling.
Preparing for the race
We planned the race day for about a week after the initial IA Guild kickoff, hoping to get around 10 teams of two or three people to have a go at training their own model. The response was strong enough that we quickly had to cap the number of teams at 15, some of which had four or five members. It was also positive to note that not just “technical” people joined in, there was a real cross-section of the company, all the way up to the leadership team who were keen to get involved.
We use AWS Single Sign On to provision our AWS access so getting our racers on-boarded was pretty simple for the most part; a new DeepRacer permissionset with a minimal IAM policy gives just enough access to create models, train, and evaluate them without affecting any existing accounts or permissions. So we did this and gave everyone access to a single account.
Obviously, this is the part where we’d love to say that everything went super smooth and without any problems at all. To be fair, it was pretty smooth, but no event ever goes perfect so we had our share of tiny hiccups.
The Create Model page is where we came across our first little hurdle. AWS helpfully provides a Reset Resources button to “[reset] the resources created by AWS DeepRacer to start again from a clean slate”. With a single user working in their own AWS account, this button is useful to quickly fix any issues with the DeepRacer setup.
When there are 50 people (with varying amounts of AWS experience) trying out something that might be outside their area of expertise, this button is a liability as it terminates any running training sessions. It also can’t be removed from that model page, and it’s the first big button you see when you create a model.
Our single account approach caused a couple other issues as well. First, there’s a hard-limit on how many models can be trained at a time (3). There’s also no support request category to increase this number, so it seems like you have to scale at the account level (how’s that for horizontal scaling?). As it got closer to the day of the race, this became a blocker for a number of teams, and is something we’ll want to address in future.
Additionally, there are no permissions or restrictions you can set on a per-model basis, so any team can inspect and “borrow” aspects of another team’s reward function. While this might fit into the spirit of cooperation we like to foster at work, it’s hardly ideal for a competitive environment.
Let’s race!
But enough of the boring stuff, let’s fast-forward to the actual event.
Race day arrived, and with many thanks to the Melbourne AWS User Group for the loan, we had a track! It’s heavy and unwieldy, and measures about 8m x 5m, so we gave ourselves plenty of time to get it set up.
This turned out to be a Very Good Decision™, as for some reason the office has air vents in the floor. This airflow was enough to turn our track into a somewhat mountainous course, which, while fun to see, wasn’t quite what we want for a race. So we had to pull up the heavy track again and tape over these vents to prevent it from upsetting the models that our teams had trained.
We were fortunate to have a number of couches in our arena we could use as walls for the track, which helped us prevent runaway cars as well as giving a somewhat static background for the DeepRacer to work with.
A quick practice run with a demo model confirmed that the racer and track were working just fine, then it was onto the real deal. We gave each team eight minutes in which they would attempt to complete three full laps, and recorded their times as well as the number of times their car went off track and needed to be reset. No additional time penalty was given for resets, we figured that the time lost chasing down the car and getting it back on the track again was penalty enough for any waywardness!
Our 15 teams spent over 200 hours of compute time training 133 models for the race. Some teams had over 30 iterations of their model, others decided they were happy with their training after two or three refinements to their reward function and parameters. The best time on the day was just over seventeen seconds (17.09), with a number of other teams also making it into the low twenties. A break for beer and pizza (and a bit of a boost for the battery) kept spirits high amongst the competitors, and it was really positive to see a number of people who weren’t aware of the event coming by and getting curious about ML and AI.
Like all live events involving any sort of showcase, we had a number of issues during the race, although thankfully nothing serious enough to prevent everyone from getting their laps completed. Most of our problems involved the camera on the DeepRacer, which is the forward-most point on the car. This meant that it was involved in most of the contact whenever the car crashed into one of our barriers.
As this particular car (owned by Arjen) may have had its fair share of crashes, the camera now has the tendency to get dislodged whenever it crashes at speed. And while the car was tuned to not go extremely fast, it’s still a race so none of the crashes happened slowly. Whenever a crash happened this needed a full restart and reinitialisation of the car, before it could once again send images to be processed by the model. This meant we spent quite a bit of time waiting for the car to boot instead of racing.
These extra reboots also played havoc with the battery of our DeepRacer. We originally planned to have a final lap from the top four teams to see if anyone could improve upon their heat times, but ran out of power after the first of these hot laps and had to stick with the original times.
All that said, what really mattered was that everyone had fun. And we certainly did. While there was a competition, everyone wanted to see people succeed. This meant that there were cheers from the entire crowd every time a complete lap was managed (not always an easy thing), and these only grew bigger when a new record was set or a lap was completed without going off-track.
Lessons learned
Of course, we also learned some things running the race and figure it might be useful if you’re aware of those if you want to run your own event.
In an ideal world we would have had multiple DeepRacers to swap in after crashes or between teams. This would have doubled our battery life, and means there would have been more time of cars driving on the track. Or at least, have a second set of batteries. Unfortunately, it’s still unknown when DeepRacer’s will be available for purchase, and the Amazon store listing indicates they won’t ship to Australia in any case.
Having everyone train in the same account is also not great. Unfortunately, having an account per team means a lot of additional overhead as well, so we’re not quite sure what a good solution is to that.
Make sure you have everyone’s models loaded on the car several hours before the event. As the car has to be on during the loading of models, this will use up some of that precious battery power as well. And with a single car you really don’t want to spend time loading the models during the event.
Last, but certainly not least, if you run the race ensure that you have a young pair of legs. Chasing after a small car, and putting it back on the track, for hours on end has some effects the next day(s).
Again though, the most important part is to have fun, as everyone else will have that too. We certainly had fun, and in the end this race was such a success that Belong is now considering setting up their own internal DeepRacer competition. Having it be appreciated as much as it was, means that, from both a professional and personal perspective, we feel that maybe it was us who won this race.
