Deep Multi-Agent Reinforcement Learning with TensorFlow-Agents

Recent advances in TensorFlow and reinforcement learning environments, such as those available through OpenAI Gym and the DeepMind Control Suite, have allowed for rapid prototyping, experimentation, and deployment of reinforcement learning applications across many domains.

TensorFlow-Agents, a TensorFlow-2-based reinforcement learning framework, is a high-level API for training and evaluating a multitude of reinforcement learning policies and agents. It enables fast code iteration, with good test integration and benchmarking¹.

Image Source

This article illustrates the application of tf_agents to Multi-Agent Reinforcement Learning (MARL) problems. In this article, we apply tf_agents to our novel, multi-agent variant of OpenAI Gym’s CarRacing-v0 environment. Our implementation of this MultiCarRacing-v0 environment can be found here. For more information on this environment, please check out this article.

Figure 1: Our novel OpenAI Gym racing environment, configured for multi-agent racing.

Algorithm: Multi-Agent Proximal Policy Optimization (PPO)

To train our agents, we will use a multi-agent variant of Proximal Policy Optimization (PPO), a popular…