AI: OpenAI Gym
OpenAI has been on the news for some time. The media attention of OpenAI is deeply related with the interest in Reinforcement Learning (RL) applications.
In the graph below we see the google searches related with RL and OpenAI. That is not a coincidence since OpenAI offers a tool used by many in the field, which is the OpenAI Gym.
In this article I go over what is the Gym and how to use it in your applications.
What OpenAI Gym is
As stated on their website, the main purpose of the gym is provide environments so that people can try their own Reinforcement Learning algorithms against them.
The difficulty of RL task can greatly vary depending on the observation space. For instance, in the classic Breakout game, one can choose to give the algorithm the position of the paddle and the position of the ball. On the other hand, it is also possible to give simply the entire image of each frame, as it was it was done by Google DeepMind’s team. The latter is a much harder task.
For that matter, it is important to have an environment against which people can compare the results of their algorithms. The Gym offers exactly that.
How to use the Gym
You can install the library by doing
pip install gymIt has several pre-made environments that you can play. A list of the available environments can be seen on their website. Probably the most famous of the envs is the CartPole. Most of the RL libraries that you find online use it as an example.
The environments offer a simple standard interface that is composed by:
observation, reward, is_done, info = env.step(action)
This method performs the action given and then observes the resulting state returning it to you together with the any reward that was received. It also returns a true or false variable that represents if it reached an end state. For example, if you won or lost your game and there is no other action you can take. The last object is used to hold some additional info for that environment
observation = env.reset()
Resets the environment to the initial state and returns it to you.
env.render()
Renders the environment so you can visualize what is happening. In the case of the CartPole, it will create a window just like the one below
How to implement your custom environment to use with RL libraries
You might find several interesting RL libraries on github with some state of the art algorithms you want to try such as Stable Baselines. A lot of these libraries standardized on using the the OpenAI Gym interface, therefore, to effectively use them, you will have to adapt your custom environment.
To adapt, you have to implement a class that inherits from the base gym.env class and then implement the methods step, reset and render.
The class below can be used to guide your efforts.
Final Thoughts
OpenAI Gym is a very useful standard for Reinforcement Learning.
It allows to easier comparison of algorithms using their available interfaces.
If you are building a Reinforcement Learning framework, following the Gym standard makes it easier for people to interface with your tool and not have to care about millions of ways someone might decide to interact with environments.
Let me know what you thought about the post. Check me out on any of these places:
