AI for video games

At WildMeta we want to bring machine learning techniques to modern video games.

In 2019, OpenAI Five defeated Dota 2 world champions with their reinforcement learning-based AIs. The same year, DeepMind applied a similar approach to create AlphaStar and achieve Grandmaster level at Starcraft II.

Inspired by these works on competitive video games, we realised that there was more than beating world-class players and that AI could be used to improve players’ experience and help game developers.

For the past few months, we have been focusing on developing our core technology. We used Dota 2, one of the most popular and complex esports games, as an environment for our machine learning-based bots. However, we didn't want to limit the scope of our work to this game, so we designed our system for reusability in addition to speed and scalability.

In the video above, our bots either play Shadow Fiend (Radiant side) or Queen of Pain (Dire side) in 1v1 against a hardcoded bot. We chose these two Dota 2 heroes as they have distinct characteristics and their respective initial ability behave differently: Shadow Fiend needs to place itself well with respect to other units (right distance and orientation) to hit them, while Queen of Pain needs to select a target in range. Our bots quickly learn to position themselves so that they benefit from experience generated by other units while staying away from the opposing hero. They also learn fundamental techniques such as last hits using either attack or ability, which grants extra experience and gold, as well as denies, which prevents a last hit from the opponent hero and gives our bot more experience. More interestingly, they learn to lure enemy creeps under an allied tower so that it decreases their health and then last hit them, use clever positioning in order to draw aggro on other units, and to navigate back to their fountain to regenerate their health and mana.

This demonstration has been designed to show what we can do with our technology on a small scale. These behaviours have been learned only after several hours of training with a regime that allows learning fundamentals as fast as possible. Training for longer (a few days) allows the bots to refine their techniques and become better opponents.

We can imagine many different use cases based on our technology:

• Human-like AIs: natural interactions with players, better in-game engagement.
• Learned navigation: no explicit pathfinding, navmesh, checkpoints, obstacle avoidance code.
• Training bots: help players improve at competitive/cooperative esports games.
• Always available opponents: bots with human-like playstyles to always have opponents to play with in multiplayer games.
• AI teammates: cooperative behaviours, behaves rationally while following orders from players.
• QA/Testing: automatically find bugs by exploring the environment, multiplayer server stress-test.

For more details on the technical aspects of our technology, we’ll be sharing another post in the coming days, so be sure to follow us on Medium, Twitter or LinkedIn.

And do not hesitate to reach out at contact@wildmeta.com.

WildMeta, AI for video games.