List of Reinforcement Learning Environments
I’ve been lately working with Reinforcement Learning (RL) and I have found there are lots of great articles, tutorials and books online about it, ranging from for absolute starters to experts on the topic. However, I’ve found the hard way that spotting a good RL environment to test your RL algorithms can take several hours. This is why I decided to write this blog post — my first blog post indeed — , to try to make a quick guide of ready-to-use RL environments available out there, that hopefully it will save some time to others (and future me!).
If you are still thinking if RL is the right method to solve the problem you are dealing with, feel free to check my post “When to use RL (and when not to).”
There are lots of RL environments you can use for your projects and many of them are open-source. Moreover, most of them have implemented an interface which resembles the OpenAI Gym environments, making them even easier to use. In case you are wondering what is this, OpenAI Gym is a collection of RL environments developed by OpenAI, which lets you play and compare your RL implementations and is broadly used by RL developers and researchers alike. You can check below for more information.
Having said this, let’s just head on to the list of RL environments:
OpenAI Gym — Atari games, Classic Control, Robotics and more
This is a wonderful collection of several environments and is heavily used by the community. You can find Atari games, classic control problems (e.g. Cart-Pole), algorithms (e.g. teach the agent how to copy text), MuJoCo for continous control tasks (e.g. teach a spider how to move in a physics environment), robotics (i.e. teach a task to a robot arm) and text games (e.g. blackjack).
License: MIT License
Official Website | Github
Gym Trading — Trading
If you want give a try to RL for trading Gym Trading is a very good option. This is another environment implemented by OpenAI and provides daily observations based on real market data pulled from Quandl on, by default, the SPY etf.
License: MIT License
Github
TensorTrade — Trading
If you’re looking for something more advanced and highly customizable, TensorTrade has got your back. This framework focuses on being highly composable and extensible, to allow the system to scale from simple trading strategies on a single CPU to complex investment strategies run on a distribution of HPC machines.
License: Apache License 2.0
Github
VIZDoom — Games [Doom]
This brings so many memories. With VIZDoom you can teach a RL agent to play the well-known and beloved Doom. VIZDoom has been used in many research papers, it’s well documented and well maintained.
License: MIT License for VIZDoom code, ZDoom’s code uses different licenses
Official Website | Github
OpenSpiel — Games
A collection of environments and algorithms developed by DeepMind, for research in general reinforcement learning and search/planning in games. OpenSpiel also includes tools to analyze learning dynamics and other common evaluation metrics. It includes several games such as Backgammon, Chess and Go.
License: Apache License 2.0
Github
ns3-gym — Networking
The network simulator ns–3 is the de-facto standard for academic and industry studies in the areas of networking protocols and communication technologies. ns3-gym is a framework that integrates both OpenAI Gym and ns-3 and provides multiple networking problems such as traffic control, in which you can test your RL algorithms.
License: GNU General Public License v2.0
Github
RecoGym — Recommender Systems
There have been many works lately that apply RL to recommender systems, usually as multi-armed bandit problem. RecoGym is an RL environment for recommendations for e-commerce advertising.
License: Apache License 2.0
Github
OpenSim-RL — Biomechanics
Design artificial intelligent controllers for the human body to accomplish diverse locomotion tasks. This library has been implemented by Stanford and annual competitions are held, in which RL practitioners can put their skills up to a test against each other. Three environments are currently implemented: a simplified arm movement, learn to run and leg prosthetics.
License: MIT License
Official Website | Github
Textworld — Text-based games
If you haven’t found the right environment for you yet, you can also easily create a RL environment to match your needs using Textworld. Textworld is not an environment per se, but a generator of environments instead. It’s implemented and maintained by Microsoft and the community, well documented and easy to use.
License: MIT License
Github
This is all for now, but I will keep adding environments in the future. If you have any suggestions about an environment to include in this list, I’ll be more than happy to read you in the comments. Thanks for reading and I hope you found this useful.
