Dev-Containers: Better Development Experience

A development container (or dev container for short) allows you to use a container as a full-featured development environment. It can be used to run an application, to separate tools, libraries, or runtime needed for working with a codebase, and to aid in continuous integration and testing.

The first thing any developer will do before starting a new project or start working on existing project is to setup the environment for that project. It may be env variables, libraries, tools or other dependencies. In an existing project the dependencies to start a project may be even greater as multiple different libraries, tools, or other components may be needed to be installed on local system.

README can help, but sometimes README just doesn’t work. You may use scripts to set these up but it may cause different issues.

The machine setup may also be different , you may be using systems on linux windows mac or using processors based of amd or arm . the combinations are huge. Keeping scripts for all the systems is a challenge and you may end up supporting a few and hoping it will work on rest of systems.

Not only this, while setting up for project its also possible to break the existing setup by downloading or setting up some dependency which conflicts with existing system on your system. It can be conflicting versions, different library modules etc.

Apparently these problems are already solved by existing technology called containerization.

A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another. A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings

And that's where dev-containers come into the picture, dev-containers provide you a way to build and run your dev environment inside a container.

This micro blogs helps you to get started with dev containers with existing or new code base in vscode. Finally we will also setup dev containers for vim.

Prerequisites

You should have docker installed on your local.

Architecture

Lets build some intuition on how can we build a similar application like Dev containers.

To build a system similar to dev container with limited capabilities, we can

1. First we need an container which has a code editor built in and some mechanism to expose the container outside with a frontend.
2. Then we can mount our main source code with workdir inside the container. This will allow you to change code from inside the container.
3. To support different features, we can use dockerfile to create and setup the development environment

We will be using vscode which already has dev container support built in

https://code.visualstudio.com/docs/devcontainers/containers

Dev Containers can be also used to experiment and CI CD. Below loops determine the shared components between inner loop(during development) outerloop(during CI/CD or after development) and production containers.

https://containers.dev/overview

Editors which currently support dev containers can be found here

Setting up Dev container in VS code ( QuickStart)

In VS code dev containers are available with extensions.

Press ctrl+P and then > to open up commands

Search for devcontainers

Lets create a new devcontainer

Choose New Dev Container, and the environment you want to use as a base.

I am using alpine as base but you can choose from python to javascript depending on your usecase.

Click on additional options to configure the env further or create using default settings

You can add more customizations to the dev container using prompt.

You can add various components like password manager, bash and utilites and other components to you container

Click Ok and vs code should open your workspace in the container after some time

Note: Now you are working inside your isolated container. While using terminal utilities you have installed may not be available inside the container. also utilities you install will not be available outside the container.

Any changes made in workspace will be available once you close session

Your container is ready , you can check it out in your main shell (remember now your vs code is using container env so you have to use other shell not vs code terminal)

In your workspace, a file would be added under .devcontainer dir

This file can be used to recreate and share the Dev environment.

Lets Explore the file

// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/alpine
{
 "name": "Alpine",
 // Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
 "image": "mcr.microsoft.com/devcontainers/base:alpine-3.18"

 // Features to add to the dev container. More info: https://containers.dev/features.
 // "features": {},

 // Use 'forwardPorts' to make a list of ports inside the container available locally.
 // "forwardPorts": [],

 // Use 'postCreateCommand' to run commands after the container is created.
 // "postCreateCommand": "uname -a",

 // Configure tool-specific properties.
 // "customizations": {},

 // Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
 // "remoteUser": "root"
}

Here

name: it is name of your environment.

image: image to use for dev container. you can customize the image section to use custom images.

features: add extra features like utilities to the dev container

postCreateCommand: if you rebuild the container, you will have to reinstall anything you’ve installed manually. To avoid this problem, you can use the postCreateCommand property in devcontainer.json or a custom Dockerfile.

A custom Dockerfile will benefit from Docker's build cache and result in faster rebuilds than postCreateCommand. However, the Dockerfile runs before the dev container is created and the workspace folder is mounted and therefore does not have access to the files in the workspace folder. A Dockerfile is most suitable for installing packages and tools independent of your workspace files.

The postCreateCommand actions are run once the container is created, so you can also use the property to run commands like npm install or to execute a shell script in your source tree

There is also a postStartCommand that executes every time the container starts. The parameters behave exactly like postCreateCommand, but the commands execute on start rather than create.

Now Most of time we would have an existing project. lets see how can we onboard that to devcontainer

Adding variables

You can add variables by adding them directly config

"containerEnv": {
    "MY_CONTAINER_VAR": "some-value-here",
    "MY_CONTAINER_VAR2": "${localEnv:SOME_LOCAL_VAR}"
},
"remoteEnv": {
    "PATH": "${containerEnv:PATH}:/some/other/path",
    "MY_REMOTE_VARIABLE": "some-other-value-here",
    "MY_REMOTE_VARIABLE2": "${localEnv:SOME_LOCAL_VAR}"
}

You can also use .env file

First, create an environment file somewhere in your source tree

Next, depending on what you reference in devcontainer.json:

Dockerfile or image: Edit devcontainer.json and add a path to the devcontainer.env :

“runArgs”: [“ — env-file”,”.devcontainer/devcontainer.env”]

Docker Compose: Edit docker-compose.yml and add a path to the devcontainer.env file relative to the Docker Compose file:

version: '3'
 services:
 service-name:
 env_file: devcontainer.env
 # …

Docker compose will auto pickup .env folder in same folder

Dev Containers and Dockerfile

instead of image we can use build and provide args to build the env with dockerfile

// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/alpine
{
 "name": "Alpine",
 // Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
 "build": { "dockerfile": "Dockerfile" },

 // Features to add to the dev container. More info: https://containers.dev/features.
 // "features": {},

 // Use 'forwardPorts' to make a list of ports inside the container available locally.
 // "forwardPorts": [],

 // Use 'postCreateCommand' to run commands after the container is created.
 // "postCreateCommand": "uname -a",

 // Configure tool-specific properties.
 // "customizations": {},

 // Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
 // "remoteUser": "root"
}

Dockerfile provides flexibility of building current codebase with existing image. for example if you want to build a python application with some existing image example-app-python. you can use it as base for your dockerfile.

This is how you can use dev container to create a workspace inside a container.

To share dev container, you can share .devcontainer folder. In vs code you can do ctrl+p then type > , select reopen in container to open the workspace in container.

But not only this you can use dev container to even setup a small env with different tools

Lets say your app requires sql database alongside main code for testing. We can use docker compose for setting up the enviroment

Dev Containers and Docker Compose

you can add following to use docker compose

"dockerComposeFile": "../docker-compose.yml",
 "service": "the-name-of-the-service-you-want-to-work-with-in-vscode",

It makes running services and mounting different volumes easy

ref. https://code.visualstudio.com/docs/devcontainers/create-dev-container#_use-docker-compose

if you want anything running in this service to be available in the container on localhost, or want to forward the service locally, be sure to add this line to the service config:

# Runs the service on the same network as the database container, allows "forwardPorts" in devcontainer.json function.
network_mode: service:db

Extend your Docker Compose file for development

Referencing an existing deployment / non-development focused docker-compose.yml has some potential downsides.

For example:

Docker Compose will shut down a container if its entry point shuts down. This is problematic for situations where you are debugging and need to restart your app on a repeated basis.

You also may not be mapping the local filesystem into the container or exposing ports to other resources like databases you want to access.

You may want to copy the contents of your local .ssh folder into the container or set the ptrace options described above in Use Docker Compose.

multiple docker compose files can be used to run different components

You can find full docs here for vs code

Saving the State And Experimentation

While working on containers are enticing one problem one may face is ephemeral nature of containers. When you are working with containers its totally possible the you download some tool or dependency specific to your requirements or experimentation. Problem arises when you close the container all changes made are lost and during next run you have to reinstall your components.

You may add them to dockerfile or postcreate command but doing that temp may become a hassel. One way to circumvent this is saving the state of docker container into the different container and using that as base image

You can use following command to save the version of image locally. Giving tag latest you can always pull last change made.

docker commit <container-id> imageName:imageTag

Once you are done testing or experimenting, you can add dependencies to main dev container docker file.

Ref.

Devcontainers: https://github.com/devcontainers

Official Website: https://containers.dev/

Guide: https://docs.github.com/en/codespaces/setting-up-your-project-for-codespaces/adding-a-dev-container-configuration/introduction-to-dev-containers

Further Read:

Developing inside a Container using Visual Studio Code Remote Development

Dev Container Guides

Hope you liked the article, If there are any suggestions or mistakes i have done please let me know.