Building, isolating and replicating development environments in Python

Photo by Max Nelson on Unsplash

Python is a language that is getting more popular than ever as it offers a lot of the things that make it a very productive tool. It’s fairly easy for developers to just fire up their text editor and start coding in Python. But there’s even more: in order to take full advantage of the vibrant ecosystem the community had built around it, a few prerequisites need to be taken care of.

More than a language

Python is more than just a language; thanks to vibrant and very active community, a rich ecosystem was built around it, with so many useful packages. These packages make our lives easier and our work more productive as these pre-made bits of code can be used to enhance our own. If you’re building an app that displays trains schedules, you won’t need to build the dates and time functionality yourself; You can either use Python’s default library or, if even that doesn’t cut it, download a third party package like Arrow to deal with it.

To make things even better, Python has a tool to manage all these dependencies, and their own dependencies, in a human-friendly manner: PIP (which stands for PIP Install Packages!). All one has to do is to tell pip what library to install (and, optionally, what version of it) and the good plumber will take care of the rest. Well, almost.

But first, let’s see how to install PIP and how to use it (As I’m an Ubuntu user, all the examples here are suited towards Canonical’s OS, if you’re using another Linux distro, MacOS or Windows, … some of these instructions will not help you).

Installing PIP is one command away:

$ sudo apt install python3-pip

This tells Ubuntu to got fetch the PIP version built for Python 3, the only and one version of Python you should be using for you new projects.

Now that PIP is here, let’s summon it to install Django, a very popular web framework:

$ pip install django

Our package manager would the look around him, find the latest version of Django and all of his dependencies, and install them all on your machine.

Please, refrain from using PIP to install packages for the time being. We’ll get to it in a moment.

Separating development environments

It all sounds awesome until you realize that, for some reason, now you need to install a different version of Django on the same machine. Wouldn’t be nice if it were possible to install each version independently for each project?

Of course, you can. You just need to add one more (easy to use) tool to your toolkit: virtualenv.

Our new little friend enables us to create isolated virtual environments where each one is well separated from the others with its own version of the needed packages … and its very own version of Python!

The beauty of virtualenv is that it allows you to toggle in and out of environments depending on what you need. You are in charge!

Once virtualenv is in place, as we’ll see below, you can start fiddling with PIP in peace. Without further ado, let’s get it done:

$ pip3 install virtualenv

and… that’s it! You now have it!

Creating isolated environments is very easy too. Here’s an example:

$ virtualenv ex_env path/to/env

Where ‘ex_env’ is the name of the environment you want to create and you can name it whatever floats your boat. I, for one, prefer to match the names of the project and its virtual environment so that I can easily tell which is which.

The path/to/env is the path to directory where you want to save all the files needed. You don’t have to worry about the contents of the directory, you have just to choose where to put it.

Does it have to be in the same directory as the project itself? No, it doesn’t. In matter of fact, I put my envs in a folder aptly called envs in my home folder, that is ~/envs/. But if you wish to save the env folder in the same directory as the project, just remember to add it to your .gitignore file (or whatever your VCS uses for that).

Creating an environment is the first step. Next: activating it so you can install all the packages your project need, and run said project. To do so, all you need is this single command:

$ source ~/path/to/env/bin/activate

As you can see, choosing a memorable, easy to type path for you envs is quite vital as you need to type it every time.

If the activation went through successfully, you’ll notice that the command prompt now has $ (env_ex) at the beginning of the command line prompt, where ‘env_ex’ is the name you chose for your environment.

Now, whenever you install a new package, you can limit its scope to a tightly isolated environment. No conflicting packages would collide again.

To quit an environment, just type:

$ deactivate

Done!

Rebuilding development environments

Combining PIP and VirtualEnv has a lot of advantages. It can for example allow you to replicate development environments easily and error-freely. Why would we need such a functionality? For all sorts of things: uniforming development environments across many machines, migrating from an older machine to a new one, …

But before replicating an environment, we need first to take a snap of it current state. This is so easy, it can be done with one single line:

$ (ex_env) pip freeze >> requirements.txt

This tells pip to freeze the current configuration, i.e. to print the currently installed packages in the current environment. The >>command is how to tell Linux to not display the output of the left side command and instead to put it inside a file called requirements.txt. As far pip is concerned, the file can be named whatever you want it to. It’s only a matter of convention to call it the way we do, and as I’m sure you already know, conventions are better be followed when coding.

When the previous command finishes executing, we end up with a new requirements.txt file containing a list of all the packages installed in the current environment.

The next step is to head to where we want to replicate this environment, be it in the same machine, a machine in the same room, or in a machine in a galaxy far, far away — as long as Python and Pip are installed.

Once there, we just type:

$ (another_env) pip install -r requirements.txt

Which tell our Pip to fetch the list of the packages and to install them, and their dependencies, to recreate the (almost) exact same environment.

And voilà!