How to solve the “it works on my machine” problem using dockers

If you have ever worked on python applications, you might be familiar with the frustrating situation of “it works on my machine” problem. This is when your code runs perfectly on your own system, but fails to run on someone else’s system. This problem becomes even worse when you have to develop your code for different operating systems, or when you collaborate with other developers who use different operating systems. How can you ensure that your code works consistently and reliably across different environments? The answer is docker.

Docker is an open platform that allows you to build, deploy, and run applications using containers. Containers are self-contained, lightweight, and portable modules that include your applications and their requirements. With docker, you can create your application on a specific OS, such as ubuntu, that runs inside a docker container with a fixed configuration. This ensures that your python application will run consistently and reliably across different environments. This is because your application is isolated from the host OS by the docker container.

You may be thinking that learning docker is too much hassle for you, especially if you already have a lot of work to do. Don’t worry, this article is not going to teach you docker from scratch, as there are plenty of excellent resources online for that. Instead, I will share with you a python package that I have created based on my years of experience with docker. This package will help you use docker easily and quickly without having to know all the details and commands. However, if you want to master docker, you can check out this comprehensive course on Udemy by Mumshad Mannambeth.

Installation of docker desktop

To get started, we need to install docker desktop on our machine. Installation of docker desktop is straightforward with one-click-install application available for your Mac, Linux, or Windows environment at their official site here.

Installation of ViperD package

viperD is a package that simplifies working with docker. I created this package based on my experience with docker. You can easily install it using pip with this command

# you can install the package using pip as follows 
pip install viperd

Using viperD to create a new project

After installing viperD, you can take advantage of its features to set up a directory structure for your new project. All you need to do is run the command “viper new” and enter your application name when prompted.

$ viper new 
Your application name: your_project_name

Understanding the directory structure created by viperD

The command “viper new” will generate a directory structure for your application that looks like this:

directory strcture created by viper new command

code_mount — This is where you will store your python application code.

data_mount — This is the folder where you will keep your data. You can choose a different folder for your data, but I prefer to keep it separate from the code. Also, it is essential to store your data/code in these folders only as they are linked to the docker container. This will make sure that your changes are saved when you stop the docker container. I will explain more about this in a later section of the article.

Dockers_file — This is the folder where you will see the docker files for creating docker images. You can ignore this for now, as the scripts in the execute folder will take care of it for you.

execute — This is where you will find the scripts for automatically building docker images and running containers. Depending on your system OS, you can use the scripts in Windows or Linux (or Mac) subdirectory.

Understanding Scripts in the execute folder

These are the scripts in the execute folder —

• build_images_initiatory —this script is for building your first docker image based on the docker files in the “Dockers_file” folder.
• build_images —this script is for updating your docker images when you modify your docker files in the “Docker_file” folder
• run_docker_dev.sh — this script is for running your docker container using the docker image built by build_images_initiatory / build_images script.

Running docker container

Now let us see how to use this package for your new project. Go to the execute folder and choose the subdirectory windows/linux based on your OS. When building your first docker image, we have to run the following script

# use this command when running on Linux/Mac OS 
source build_images_initiatory.sh

# use this command when running on windows OS
build_images_initiatory.bat

After the docker image is created, you should run the following script.

# use this command when running on Linux/Mac OS 
source run_docker_dev.sh 

# use this command when running on windows OS
run_docker_dev.bat

After running the above command, you will enter the linux shell of your docker container. Now you can execute your python code in this terminal. For example, if you have a python script named app.py, you can run it as

root@3e6fdc902c5b:/your_project_name/code_mount# python3 app.py

This is how you can use docker to run your python application. Now you may realize that your python script will run consistently and reliably across different environments, as it is always running on ubuntu inside the docker container. Also, you don’t need to create virtual environments for your projects, as you are using different directory structures and docker containers are isolated from each other and the host OS.

Some addition information

Here is some extra information that will be useful when you use this package.

Installation of python packages

Only the code and data folders are permanent when running docker as they are mounted from your host system to the docker container. This means any changes you make to the code/data directory will be saved even when you stop or restart the docker. All other changes like adding new folders or packages will not be saved. But don’t worry, here is how to install packages. When you are running docker and experimenting, you can install packages using your normal pip command. Later you can add these packages to the requirements.txt file under Dockers_file>requirements folder.

When you stop the docker container, you will have to run the script “build_images”. This script will install all the packages in the requirements.txt file and update your docker image accordingly. This way, you will have these packages available when you run your docker container next time.

Installation of Linux packages

If you need to install linux specific packages that pip cannot install. You can write the commands for installing these packages in the “Dockerfile” file under Dockersfile>app_initiatory folder. Then, you will have to run the script “build_images”. This script will install all the packages and update your docker image accordingly. This way, you will have these packages ready when you run your docker container next time.

Running of python specific commands

Some python packages need you to install certain model files after installing the package. For example, in nltk you have to run this command to download standard stopwords.

import nltk
nltk.download('stopwords')

For these cases, you can paste these commands in requirements.py file under Dockers_file>requirements folder and run the “build_images” script.

conclusion

I hope you enjoyed this article and will use the viperD package for creating your new python applications. Please share your feedback in the comment section and I will try to address any issues as soon as possible.