Technology Fridays: OpenAI Gym Makes Reinforcement Learning Real
Welcome to Technology Fridays! Today we are going to artificial intelligence(AI) land with one of the most creative technology stacks in the agent enablement space: OpenAI Gym. The first product released by the OpenAI Foundation(Elon Musk, Sam Altman…), OpenAI Gym is a framework for the implementation and evaluation of reinforcement learning algorithms.
Reinforcement learning is becoming one of the most exciting disciplines in modern AI solutions. conceptually, reinforcement learning focuses on building knowledge in AI agents using a combination of rewards and punishment. Some experts might argue that all forms of AI learning can be modeled as variations of reinforcement learning but, as an AI discipline, reinforcement learning has taken a back seat to other forms such as supervised learning. However, the recent emergence of AI scenarios that operate in incomplete environments such as self-driving cars or even poker games have triggered a new interest in reinforcement learning.
One of the biggest challenges in reinforcement learning applications is the need to regularly and efficiently train and evaluate models. This is the challenge that OpenAI Gym addresses. The platform provides the foundational pieces to compare and evaluate reinforcement learning models. At a high level, OpenAI Gym consist of two main components: OpenAI Gym Library and Service. The OpenAI Gym library is an open source framework that provides a collection of environments and test problems that can be used to evaluate reinforcement learning algorithms. The OpenAI Gym service is an API and web interface that enables the comparison of the performance of reinforcement learning models.
Environments are a concept at the core of the OpenAI Gym architecture. From the AI theory standpoint, Environments provide an abstraction to the outside world surrounding an AI agent. OpenAI Gym includes a large collection of environments abstracted by the Env interface. the platform also includes a registry that lists available Environments associated to classic AI problems such as Atary, Inverted Double Pendulum, Go9x9 and many others. Each OpenAI Gym Environment exposes its specific requirements via the EnvSpecs interface.
OpenAI Gym Environments provide operations that enable their interaction with AI agents. For instance, the Reset operation restarts an Environment to its original state while the Step operation advances the Environment by one timestep. Developers can use these operations to interact with Environments from their reinforcement learning models.
There are no hard specifications to OpenAI gym agents. the platform an open to any reinforcement learning algorithms. Typically, the role of an agent is to send to data Environments and receive back a list of observations and rewards.
One of the coolest capabilities of OpenAI Gym is the evaluation of AI models. Every time a model runs, the results can be uploaded to the OpenAI Gym servers via its APIs. Some models are automatically scored while others require peer reviews. The platform maintains a list of evaluation associated with specific environments.
OpenAI gym is a new platforms in a nascent field knows as AI agent enablers. Despite OpenAI Gym unique focus on reinforcement learning, platforms such as DeepMind Lab, Kitt.ai or Octane.ai can also be considered relevant in the space.