Preparing a headless environment for OpenAI’s Gym (with Docker and Tensorflow)
It shouldn’t be hard, should it?
Actually, it is way hard to just make OpenAI’s Gym render especially on a headless (or a cloud) server because, naturally, these servers have no screen.
Gym needs a display (but not a screen) to be able to render many environments. The most bulletproof way I have realized is to have a “full-fledged desktop environment” that you can remote into just in case you want to see it live, preferably via VNC.
All of these could be done in a Docker container which should alleviate you from some level of dependency hell, which I think is a real concern for Gym. Moreover, I prefer Ubuntu.
So, we first find a working desktop environment on Docker as a starting point. I quickly found this https://github.com/fcwu/docker-ubuntu-vnc-desktop, which contains:
- Ubuntu 18.04 (and others you can choose)
- LXDE on Xvfb. Why LXDE? It is a very light desktop environment. What’s Xvfb? It is a virtual display provider. A headless server doesn’t have a display, this creates one virtually.
- Allow remote access via x11vnc, at its gist x11vnc is a way to host an X11 display via VNC. Basically, this will host the Xvfb display.
- Having a web client for noVNC meaning you don’t need to install a VNC client you just need a web browser (it is a nice feature, but I will not use it)
What Ubuntu version to use?
I think the most compatible is 16.04 (not 18.04). Especially, if you want to also use Tensorflow, you will need CUDA 9.0 for official support, but the CUDA 9.0 is, in turn, only officially supported up to 16.04. That means CUDA 9.0 on 18.04 is an uncharted territory.
However, there is some hiccup with the Gym. Even though my heart believes that Gym would work great with 16.04, but I tried and it didn’t work. The problem was upon .render() an environment, it hangs. I have enough evidence to conclude that whatever causes this must be in the difference between 16.04 and 18.04 (of course with a desktop environment installed) because I found 18.04 to work.
Conclusion, I will pick Ubuntu 18.04 for this article and will show you a way to survive in that uncharted territory.
Just a working Gym
With a working desktop environment, this seems trivial. What we need is to follow the instruction from https://github.com/openai/gym. In 18.04, not all the dependencies are available for exmaple libav-tools is not available in 18.04 we need to install ffmpeg instead.
The dependencies become:
apt-get install -y python-numpy python-dev cmake zlib1g-dev libjpeg-dev xvfb ffmpeg xorg-dev python-opengl libboost-all-dev libsdl2-dev swigMoreover, I tend not to rely on the system’s Python mainly because I don’t fully understand it, and also python means Python 2. If we want to use Python 3 need to always use python3 and pip3 which are cumbersome to me. Leave the system Pythons alone then! We will use a Miniconda. Fortunately, Miniconda provides an official Dockerfile we can integrate it quite easily.
I provide a modified version of Dockerfile (with no noVNC and a web client) here: https://github.com/phizaz/docker-cuda-gym/tree/master/gym-only
Notes on Dockerfiles
All the Dockerfiles are written with an assumption to always use “root” user inside the container. But, this shouldn’t mean that it must be root from the outside. Docker provides a neat way for user namespacing so that a root user in a container will be seen as a normal user outside. This makes our Dockerfiles much easier to write.
If you want it with Tensorflow
I assume that you also want a GPU version of Tensorflow which means you need to install a CUDA of some version (preferably 9.0 the latest officially supported CUDA version), also you need nvidia-docker for this.
Okay, we want to use CUDA 9.0, but as mentioned above we also want to use Ubuntu 18.04 (somehow 16.04 has a problem with Gym). This creates an awkward situation because CUDA 9.0 doesn’t support Ubuntu 18.04 (you will not find an official Docker image from nvidia). We thus need to make it happen by ourselves.
The general recipe for installing CUDA 9.0 on Ubuntu 18.04 can be found from https://gist.github.com/Mahedi-61/2a2f1579d4271717d421065168ce6a73
Generally, you need to download a “runfile” from Nvidia website. Also, Tensorflow needs CUDNN 7 to work you need to download it from the website as well.
I have provided a template for the Dockerfile. Hopefully, if you have a proper runfile and a CUDNN archive, you should be able to build a Docker image with ease.
A recipe for CUDA Dockerfile: https://github.com/phizaz/docker-cuda-gym/tree/master/cuda
Given that you now have the CUDA base image, we move on to installing Tensorflow and Jupyterlab. This should be most trivial since we have CUDA 9.0 and it is officially supported. Additionally, Jupyterlab is also available via conda install .
Here is the Docker recipe for the rest (Tensorflow + Jupyterlab): https://github.com/phizaz/docker-cuda-gym/tree/master/tensorflow
May this article save you hours of trial-and-error on building a working Gym environment.
