Building a Docker Quant Environment

How to create a local docker environment for Quant development

Published in

Quant Factory

8 min readOct 20, 2022

So you are looking at the myriad tools and packages that require you to be your baseline for developing your strategy and are wondering — how do I replicate my environment across my machines? What do I need to do to actually ship my code to a colleague so he can deploy it? And what if one of my quant tools requires a connection to a database — do I need to handle that?

Docker answers all of your questions.

I will be very grateful if you can fill out the below form.

It takes 5 seconds and it will help me a ton :)

So what is Docker anyway?

Docker is a software platform, package software into standardised units for development, shipment and deployment (lifted straight from docker’s website)

How can I get started?

To get started I would advise you to use Docker Desktop. Head to here, choose your distribution and install the executable.

Once the installation is complete you should be able to go to your command line and type:

docker -v

It’s time to run your first container:

docker run hello-world:latest

So what did we do here? We used the command run to execute the default script in the image hello-world with the tag latest.

The Docker image

The docker image acts as a template with instructions on how to create the Docker container. Many freely available ones can be pulled directly from the docker hub. Even when you are starting out you will be using a different image as a jumping off point.

To pull one from the hub, just search for the desired image and when you go to its page and execute the command.

⚠️📈💲MY FREE STOCKS RESEARCH NEWSLETTER! 💲📈⚠️

The Seeker

Do you need access to concealed and alternative data-driven research and analysis? Don't miss out on the best…

quantfactory.substack.com

docker pull <image>

If you want to run a specific tag of the image you have to add it to the docker image name with a colon.

docker pull <image:tag>

For example let pull an image, which supports python3.9:

docker pull python:3.9-bullseye

Working with images and containers

To run a pulled an image there is the command run:

docker run <image>

Lets run the python image we pulled earlier and be presented with the Python REPL.

docker run -it python:3.9-bullseye

Here the -it flag represents that we want to run the image in an interactive mode. In order to run an image in a detached mode you have to replace the -it flag with a -d flag. We will run an nginx image, to showcase it.

docker run -d -p 8080:80 nginx:latest

So whats with the -p flag. By using the flag we are forwarding the container port 80 to the external port 8080. You can browse the default nginx page at http://localhost:8080. But how will I be able to stop the running of the image?

If you run docker ps you will be shown the number of running containers and their names.

To stop a running image you have to type:

docker stop <image_name/image_id>

So in our case we can either use 1878aa523dc5 or its name priceless_brahmagupta. To view your stopped images you have to run docker ps -a. To remove the image you have to run docker rm <image_name/image_id>. So what about the images? To view your current downloaded images you can view them simply by running docker images. To remove the image you have to run docker image rm <image_name>.

How can I build my own images?

Now we come to fine fun bit. Lets build our own image that will serve as the basis of our environment:

FROM python3.9-bullseye # FROM The image, which will serve as the basis of our own
RUN pip install jupyter notebook # RUN actually specifies what command to run
RUN pip install pandas requests backtrader matplotlib # Install other dependanciesENTRYPOINT ["jupyter", "notebook", "--ip=0.0.0.0", "--allow-root", "--no-browser", "--token=''"] # What to run where we execute the image

A note here is the command to run a jupyter notebook. Besides the command, required to run it (jupyter notebook)

--ip=0.0.0.0 will make jupyter accept connections from places different places than localhost. And since we are running in a docker container, it will see a local connection as an external connection.
--allow-root will allow jupyter notebooks to execute as a root user.
--no-browser since we run in a headless docker container we don't need the browser to pop-up at start.
--token='' allows you to skip the basic token authentication when running the notebook.

Disclaimer: These settings are intended for local use only and should not be considered when running the image on an actual production server.

To continue the topic on useful commands in the Dockerfile — another useful commands are “COPY” to copy files from your local file system to the containers. For a full reference visit the docker reference documentation.

Now, assuming you are in the same folder as the Dockerfile the build command is as follows:

docker build -t <image-name>:<image-tag> .

So to give a concrete example with the case of our image:

docker build -t quant-docker-image:latest .

Running it.

So how to we actually run the image we just built:

docker run -p 8888:8888 quant-docker-image:latest

This makes the jupyter notebook accessible through a web browser at our machine at localhost:8888.

This command has one important caveat — if you stop your running container, all the work will be gone. But I want my work to be saved to my pc, I hear you say — ah so we need volumes!

Container volumes allow us to share a folder from our local machine inside the docker container and vice versa. It’s a nice way to share files between the two.

To share a folder with your container, will need to add the -v flag:

docker run -p 8888:8888 -v <folder-on-your-computer>:<folder-inside-container> quant-docker-image:latest

And you are set.

Docker Compose

So what if you need more than a single jupyter notebook to do your job. Maybe you need a local instance of a database for example. And maybe something else on top? We can use docker compose for that purpose. Docker Compose is another great tool, provided for your with the Docker Desktop.

How to create one

You need to create an yaml file, say docker-compose.yaml in your development folder. For an example lets setup our jupyter notebook container and a PostgreSQL database.

version: '3' # Define the version of docker compose we want to run atservices: # the services you intend to run  postgres: # The name of our service
    container_name: local_postgres # the name of the container that we are going to run
    hostname: local_postgres # internal resolution address of the container
    image: postgres # image to run the postgres, you don't need to pull it beforehand, docker compose will do it for you.
    volumes: # As discussed above, in order to save our database we need to share our internal folder /var/lib/postgresql
      - ./<local-folder-postgres>:/var/lib/postgresql
    environment:
      - POSTGRES_DB=quantfactory # example name of the database that postgre will create for us
      - POSTGRES_USER=quant # an user to access it
      - POSTGRES_PASSWORD=aStrongPassword # a strong password for the user
    ports: # the external port at which the services is exposed.
      - 54321:5432 
    restart: unless-stopped # the restart policy - so if the container crashes - it will restart, unless explicitly stopped.  jupyter:
    container_name: quant-jupyter # see previous service
    hostname: quant-docker-image:latest # see previous service
    build: # We supply the build command so it will build the docker image for us as well
      dockerfile: Dockerfile
    ports: # see previous service
     - 8888:8888
    volumes:
      - ./<local-folder-jupyter>:/<container-folder-jupyter>
    restart: unless-stopped
    depends_on: # When raising interdependant services you can schedule the container to run after a previous service has been started succesfully
      - postgres # We will run our jupyter container after the postgres database is up.

To run your whole stack of services you just need to run the following:

docker compose up

Easy enough, but what now your terminal is occupied with messages from docker compose. Well to run in the background (or in detached mode), just add the flag -d.

To bring the stack down, you just have to run:

docker compose down

For a more comprehensive guide and additional options, refer to the official documentation.

Wrap-up

In this guide we found out what is docker, how we can use it to build our development environment, and if we need additional services to use docker compose. Once you get comfortable with Docker, you will build more and more elaborate containers, which will help you on your way to develop and run great quant strategies.

If you have any questions, suggestions or thoughts, always feel free to share them with me! And if you like to see more how-tos on deploying your screeners and strategies, make sure to follow me and receive the most recent updates.

A Message from QuantFactory:

Thank you for being part of our community! Before you go:

Master Quantitative Research with Python👉 here.
Join our Premium Discord Server👉 here.
Subscribe to our Exclusive Stocks Newsletter 👉 here.
If you liked the story feel free to clap 👏 and follow the author
For more stories from the Algorithmic Trading and Quantitative Analysis world you should follow the Quant Factory Publication

*Note that this article does not provide personal investment advice and I am not a qualified licensed investment advisor. All information found here is for entertainment or educational purposes only and should not be construed as personal investment advice.