OpenAI’s Long Pursuit of Dota 2 Mastery

A hearty round of applause arose from the crowd packing the Vancouver Rogers Centre on August 22 when a team of unassuming scientists wearing “OpenAI” T-shirts climbed up on stage. They had come to Canada to pit their artificial intelligent bots against professional human players in a highly anticipated, world-first 5v5 showdown in one of the world’s most complex video games ever, Dota 2.

The journey to the historic match began in the winter of 2016, when an OpenAI research team led by CTO Greg Brockman was searching for a challenging game environment with competitive benchmarks where it could test its AI research and techniques against the skills of human professionals. Games are a hotbed for AI research: they are computationally complex; have rich human-computer interactions; and generate tons of data.

Founded in 2015 in San Francisco as a non-profit AI research company backed by Elon Musk, OpenAI’s ultimate goal is to build an Artificial General Intelligence (AGI) capable of performing a multitude of tasks within one general system. OpenAI regards the creation of an AI that can perform as quickly and effectively as human pros in a complex computer game environment as a major step toward achieving AGI.

Beating humans is also a convincing way for AI researchers to make their mark. The dramatic victory of DeepMind’s AlphaGo over Korean Go Grandmaster Lee Sedol in March 2016 pushed the envelope in AI gaming and secured DeepMind a place in AI history.

OpenAI researchers surveyed various games on the Twitch and Steam platforms before deciding to tackle Dota 2, which can run on Linux and has an API. Developed by the Valve Corporation in 2013, Dota 2 is a highly complex and wildly popular multiplayer online battle arena (MOBA) video game played between two teams of five players. The team that takes down their opponent’s center base “Ancient” wins the match. The game environment has 115 characters and all-important “Heroes,” 22 defensive towers, dozens of non-player characters, hundreds of skills and items, and a long tail of game features such as runes, trees, wards, and so on.

Early struggles

OpenAI’s first Dota 2 effort was a scripted computer with hard-coded rules. It could improve its tactics only by acquiring additional…