Setting a Dockerized Python Development Environment Template

In this post, we will review how to set, with a few simple steps, a dockerized Python development environment with VScode and the Dev Containers extension using a Github template.

Figure 1: Working with a containerized environment with VScode and the Dev Containers extension (Image Credit: Rami Krispin)

This tutorial required basic knowledge of VScode’s Dev Containers and Docker. In a nutshell, the VScode Dev Containers extension enables you to spin up a new VScode session inside a containerized environment. It is highly customized, giving users a large degree of freedom to customize their development environment according to their needs. If you are unfamiliar with the Dev Containers, I recommend checking this tutorial for Python or R first.

Table of Content

• Motivation
• Scope
• Prerequisites
• Getting Started
• Customization
• Resources

Motivation

Setting up a dockerized environment could be a convoluted and, in some cases, cumbersome process. Over time, I adopted different methods, such as using settings (e.g., JSON, YAML, etc.) and helpers (e.g., bash script) files to simplify the process and make it more efficient. In addition, I use VScode with the Dev Containers extension, as it provides a seamless integration with Docker. A typical project folder would have the following files under the .devcontainer folder:

• devcontainer.json — is the Dev Container configuration file that enables you to customize your VScode development environment when using the Dev Container extension. That includes the image build and run settings (similar to the docker-compose.yml file functionality) and the VScode settings (e.g., extensions to install, etc.)
• Dockerfile — the image manifest or recipe. It provides instructions for the docker engine about which base image to use and what components to install.

And the following bash scripts:

• install_dependencies.sh — install the project dependencies (i.e., command line tools, Debian dependencies, etc.)
• install_quarto.sh — install Quarto docs
• install_requirements.sh — set the Python virtual environment and install the required Python libraries using the requirements.txt files

While it is convenient to have those sets of files handy, the downside of this workflow is that creating a new project requires copy-pasting the dev container folder (which contains the above files) from an existing folder to the new project folder and then modifying the files according to the new project’s needs. This is where Github templates come into place, removing this redundant step by setting a highly customized template using environment variables and setup files. In this tutorial, we will use the below template:

GitHub - RamiKrispin/vscode-python-template: A template for a dockerized Python development…

The R version is currently WIP, and it is available over here.

Scope

This VScode template includes the following features:

• Dev Containers settings (devcontainers.json and Dockerfile files)
• Python virtual environment settings
• Quarto
• Jupyter

The template includes the following files:

.
├── .devcontainer
│   ├── Dockerfile
│   ├── devcontainer.env
│   ├── devcontainer.json
│   ├── install_dependencies.sh
│   ├── install_quarto.sh
│   ├── install_requirements.sh
│   └── requirements.txt
├── tests
│    ├── test1.py
│    ├── test2.ipynb
│    ├── test3.html
│    ├── test3.qmd
│    └── test3_files
└── README.md

Where the .devcontainer folder contains the environment settings and includes the following files:

• Dockerfile — with the image settings
• devcontainer.env — for setting environment variables
• devcontainer.json — the Dev Container extension settings
• install_dependencies.sh — a bash script that installs the Debian dependencies during the build time
• install_quarto.sh — a bash script that installs Quarto during the image build
• install_requirements.sh — a bash script that sets the Python virtual environment and installs the required Python libraries as listed on the requirements.txt file
• requirements.txt — a list of Python libraries to install in the virtual environment with pip

In addition, three test scripts under the tests folder verify that the environment’s main components are correctly set and executable.

Prerequisites

To use this template out of the box, you will need on your local machine the following settings:

• VScode
• The Dev Containers extension
• Docker and Docker Desktop (or equivalent)
• Docker Hub account

A step-by-step guide for setting the above prerequisites is available here:

GitHub - RamiKrispin/vscode-python: Setting Python Development Environment with VScode and Docker

Getting started

It is straightforward to use this template and clone it to a different account. Click the Use this template green button on the top right and select the Create a new repository option:

Figure 2: Cloning the template (Image Credit: Rami Krispin)

The steps from there are similar to working with an existing repository. You can clone it to your local machine and modify the environment configuration.

The template is ready to use out of the box with Github Codespaces, and if you have access to Codespaces, you should have the option to open the repository on Codespaces:

Figure 3: Open the template with Github Codespaces(Image Credit: Rami Krispin)

Customization

The template was created to enable seamless customization and modification of the Python environment using environment variables. That includes the Python version, the virtual environment name, installation libraries, setting environment variables, etc. Once you clone the template, you can customize devcontainer.json file according to your environment requirements:

devcontainer.json

{
    "name": "${localEnv:PROJECT_A_NAME:my_project_name}",
    // "image": "python:3.10",
    "build": {
        "dockerfile": "Dockerfile",
        "args": {
            "ENV_NAME": "${localEnv:PROJECT_A_NAME:my_project_name}",
            "PYTHON_VER": "${localEnv:PYTHON_VER:3.10}",
            "QUARTO_VER": "${localEnv:QUARTO_VER:1.3.450}"
        }
    },
    "customizations": {
        "settings": {
            "python.defaultInterpreterPath": "/opt/conda/envs/${localEnv:PROJECT_A_NAME:my_project_name}/bin/python3",
            "python.selectInterpreter": "/opt/conda/envs/${localEnv:PROJECT_A_NAME:my_project_name}/bin/python3"
        },
        "vscode": {
            "extensions": [
                // Documentation Extensions
                "quarto.quarto",
                "purocean.drawio-preview",
                "redhat.vscode-yaml",
                "yzhang.markdown-all-in-one",
                // Docker Supporting Extensions
                "ms-azuretools.vscode-docker",
                "ms-vscode-remote.remote-containers",
                // Python Extensions
                "ms-python.python",
                "ms-toolsai.jupyter",
                // Github Actions
                "github.vscode-github-actions"
            ]
        }
    },
    // Optional, mount local volume:
    // "mounts": [
    //     "source=${localEnv:DATA_FOLDER},target=/home/csv,type=bind,consistency=cache"
    // ],
    "remoteEnv": {
        "MY_VAR": "${localEnv:MY_VAR:test_var}"
    },
    "runArgs": [
        "--env-file",
        ".devcontainer/devcontainer.env"
    ],
    "postCreateCommand": "python3 tests/test1.py"
}

Note: The default setting uses the build argument with the Dockerfile and some bash helper files. Alternatively, you can use the image argument to use any other container.

The devcontainer.json main arguments:

• name — defines the project name
• build — a wrapper for the docker build command. It builds the container when launching the Dev Container extension. Alternatively, you can use the image argument to load local or external images from Docker Hub.
• mounts — optional (commented), enables to mount additional folders from the local file system in addition to the project root folder
• remoteEnv — set environment variables for the environment
• runArgs — passes arguments to the container during the run time
• postCreateCommand — executes commands on the terminal after the launch of the environment
• customizations — enables the modification of the VScode setting for the container and isolates it from the default settings. In this case, using the following two sub-arguments:
• settings — to set the Python extension default interpreter
• vscode.extensions to define the list of extensions to install upon the launch of the container

In addition, we use the following environment variables:

• PROJECT_A_NAME — defines the project name and the environment name with a default value of my_project_name. The variable is used with the name and build arguments to define the project name and virtual environment name, respectively. In addition, we use this variable with the customizations.settings argument to set the default Python interpreter path to match the venv name
• PYTHON_VER — defines the Python version by setting the base image tag, by default set to 3.10
• QUARTO_VER — defines the Quarto version to install on the environment, by default set to 1.3.450
• DATA_FOLDER — optional, used with the mounts argument to define the local folder path to mount on the image in addition to the project folder
• MY_VAR — a local environment variable passed to the dockerize environment with the use of the remoteEnv argument using a default value of test_var

All the environment variables use default values, and it is recommended that they be set locally according to your requirements.

That's it! you can now launch the environment on your local VScode. Note that the first time opening the environment, it will take time to build the image, but it caches it, and moving forward, the process should take a few seconds.

Resources

• VScode Python template — https://github.com/RamiKrispin/vscode-python-template
• Setting Python Development Environment with VScode and Docker — https://github.com/RamiKrispin/vscode-python
• Dev Containers extension — https://code.visualstudio.com/docs/devcontainers/containers
• Github template documentation — https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-repository-from-a-template

More content is available on my Telegram channel.