Setting Up a Data Lab Environment — Part 1 — I ❤️Docker

[Post is also available at quaintitative.com]

The reason I started using Docker was because I was trying to run Jupyter notebooks on Amazon Web Services (AWS). As that’s still something useful to know if all you want is something lightweight, and without the added steps of needing to install Docker, I’ve covered that in 2 previous posts:

• Basic Jupyter notebook on AWS here; and
• Jupyter notebook, with Nginx here

It’s a lot easier to do these. But to just use Docker for the two tasks above is frankly a waste.

So we what we will do in this series is to get a -

• General understanding of Docker (in this post)
• Set up a Jupyter notebook in AWS with Docker
• Learn how to use bash scripts and Docker to automate the process
• Use Dockerfiles to roll your own customised images for containers
• Set up Postgres and Mongo databases in Docker
• Set up a Flask server in Docker that can be used to show-off the data analysis we did in the Jupyter notebooks and saved to a database.

Docker

So let’s go through some basics in Docker first.

I won’t go through the full extent of what it can do or how to install Docker (just Google!), but will just run through 2–3 key commands that will help get you an understanding of what Docker does, and get you through the series.

What Docker does is quite simple. It creates a container with a barebones OS, and adds layers of software on top, depending on what you need.

Each combination of OS and software is called an image. You can use images built by others, or roll your own. Or roll your own starting with an image built by someone else.

So once you have an image, all you need to do is first get the image by pulling it.

docker pull jupyter/minimal-notebook

Then run it. The -p flag is essential to link port 8888 on your own computer to the port 8888 in the Docker container.

docker run -p 8888:8888 jupyter/minimal-notebook

When you do this, the terminal will spurt out a series of logs, and you will not be able to do anything further in the terminal. To have Docker run in a detached mode, which will allow you to continue on in the terminal, just add a -d flag.

docker run -p -d 8888:8888 jupyter/minimal-notebook

Now you have an instance of a jupyter notebook running in a container, which you can access the usual way — localhost:8888.

Usually what we would do would be to install a base installation of Anaconda, and start adding on other libraries and packages as and when needed.

With Docker, we don’t really have to do this. There are pre-defined images available that you can use, as seen in the image below which I took from here.

Image from here

For more of the most common docker commands, I find this to be a good cheat sheet.

playgrd.com || facebook.com/playgrdstar || instagram.com/playgrdstar/