CoRL 2020 Best System Paper Winner: Noah’s Ark Lab Multi-Agent RL Simulation for Autonomous Driving

The CoRL 2020 Best System Paper Award was presented today to Huawei Noah’s Ark Lab, Shanghai Jiao Tong University and University College London for their paper SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving. The CoRL 2020 Award Committee praised the work as “a thorough and well-thought-out system with strong potential impact for the Autonomous Driving community.”

The paper introduces SMARTS (Scalable Multi-Agent RL Training School), a realistic multi-agent simulation platform for autonomous driving. SMARTS supports the training, accumulation, and use of diverse road user behaviour models to help reinforcement learning (RL) researchers examine realistic road interaction scenarios. It has been open-sourced.

While exploring the open road can be exhilarating, the driving experience more typically involves navigating busy streets packed with other, often unpredictable drivers. For autonomous driving vehicles, interactions with the wide range of intelligent and not-so-intelligent other road users present a fundamental challenge with very high stakes.

The researchers identify a pain point: “Current mainstream level-4 AD (Autonomous Driving) solutions tend to limit interaction rather than embrace it: when encountering complexly interactive scenarios, the autonomous car tends to slow down and wait rather than acting proactively to find another way through.”

The team notes that in California in 2018, 57 percent of autonomous car crashes were rear endings and 29 percent were sideswipes by other cars that could be attributed to heavy breaking and the overall “conservativeness of the autonomous car.” Autonomous vehicles’ unwavering adherence to posted speed limits has also resulted in long lines of traffic on rural roads and dangerous overtaking.

Doesn’t that sort of driving sound like a typically overcautious rookie is behind the wheel? Novice drivers can head to a driving school to learn such real-world road skills, and the researchers believe RL agents should do the same, as “driving in shared public space with diverse road users is fundamentally a multi-agent problem that requires multi-agent learning as a key part of the solution.”

To make the driving simulations reflect the most diverse set of road users and scenarios as possible, besides the vehicles controlled by the autonomous driving software (ego vehicles), the researchers also paid extra attention to the simulated vehicles representing other cars (social vehicles) on the roads and their interactions with ego vehicles. Accordingly, the team created the crowdsourcing Social Agent Zoo platform to collect agents to control the social vehicles in SMARTS simulations.

The team says SMARTS realistic and diverse interactions arise out of the confluence of four key contextual factors:

• physics
• behaviour of road users
• road structure & regulations
• background traffic flow

The researchers extracted a number of insights regarding agent behaviours by running multi-agent RL experiments under three challenging driving scenarios — two-way, double merge, and intersection — that required agents to learn non-trivial interactive capabilities in SMARTS simulations.

The team says it hopes the SMARTS platform can support investigations into how various multi-agent RL algorithms perform in the autonomous driving context and guide the development of new and better algorithms. They believe the value of SMARTS will grow in the coming years, as it exposes researchers and systems to more and more realistic driving interactions.

The paper SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving is on arXiv. The team has open-sourced the SMARTS code on the project GitHub.

Reporter: Fangyu Cai | Editor: Michael Sarazen

Synced Report | A Survey of China’s Artificial Intelligence Solutions in Response to the COVID-19 Pandemic — 87 Case Studies from 700+ AI Vendors

This report offers a look at how China has leveraged artificial intelligence technologies in the battle against COVID-19. It is also available on Amazon Kindle. Along with this report, we also introduced a database covering additional 1428 artificial intelligence solutions from 12 pandemic scenarios.

Click here to find more reports from us.

We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.