Python Virtual Environments

A how-to-guide for running different Python development environments side-by-side.

In real world Python development environments, it is essential to be able to run multiple versions of the language interpreter and standard libraries, along with your application source code and custom libraries, in a way that preserves the original Python environment while also allowing different projects to be configured and run in their unique way.

For example, you might need to work on project A in Python 2.7, but another project B which takes advantage of new features in Python 3.5. Project A might also need particular modules loaded and project B a different set of modules. How can you operate in each environment without (a) contaminating the master Python 2.7 and 3.5 environments, and (b) meet project A and project B’s particular module requirements? Enter Python virtual environments.

A Python virtual environment is a sandbox which preserves the master Python environment, allows a custom list of modules to be loaded and used, and make deployment of a Python project more straightforward.

This article will outline the steps to set up Python virtual environments and then explain how to switch between them. It assumes you are starting with the default installations for Python 2.7 and 3.5, and operating on Apple’s OSX 10.6 or later.

Let’s first check your master Python environments for 2.7 and 3.5.

To locate the Python 2.7 interpreter that would be called, based on your $PATH variable, type:

> which python

To locate the Python 3.5 interpreter that would be called, type:

> which python3

These two paths will be different when you are running inside a virtualized environment, and we will use this difference to verify that the virtualized environment created is activated correctly. Just make a mental note of the paths above for now.

What is a Virtualised Python Environment?

A virtual Python environment contains a Python interpreter, the standard Python libraries, some tools to load Python modules (wheel, pip), your Python code, and any particular libraries you need to run your application.

It is a self-contained “sandbox” for development isolated from other virtual environments and the master Python environment on your machine (i.e. the one created when you installed the language). Using a virtual environment helpful when deploying your application because everything needed by your application is self-contained without dependencies outside of the sandbox.

You should use a virtualised Python environment each time you wish to develop an application in Python that requires custom modules or has to be distributed and deployed on other computers.

Step 1: Install the virtualenv2/virtualenv3 tool

To create the virtual environment, we need to make sure to load the virtualization tool into the Python environment.

To create virtual environments running the Python 2.7 interpreter we need to have thevirtualenv2 tool loaded. To create virtual environments running the 3.5 Python interpreter we need to have thevirtualenv3 tool loaded. If you install both tools, then you can create virtual environments of either Python version (2.7 or 3.5).

You will also need to decide ahead of time whether you want the virtualisation tool loaded for all users of your computer or just the current user. The example below presumes you wish to load the tool for only the current user of the computer and so includes the optional — user flag.

To load up virtualenv2, run the following from the command line:

> pip install — user virtualenv2

To load up virtualenv3, run the following from the command line:

> pip install — user virtualenv3

Step 2: Create an alias to the virtualisation tool

To make creating a virtual Python environment (either 2.7 or 3.5) easy, create aliases for both tools in your .profile. In the case of bash users (the default on OSX), edit your .bash_profile, using the vim editor with the following command:

> vim ~/.bash_profile

Press “i” to insert into the .bash_profile file and add the following lines anywhere:

alias virtualenv2=’~/Library/Python/2.7/bin/virtualenv’
alias virtualenv3=’~/Library/Python/3.5/bin/virutalenv’

Press SHIFT+zz to save and exit the vim editor and reload your .bash_profile with:

> source ~/.bash_profile

Step 3: Create a virtual Python environment

To make a virtual environment, which will contain a Python interpreter, tools to load modules (pip, wheel) and the standard Python libraries; you first need a directory to contain it all.

The directory will become the “sandbox” for the virtual environment in which all your Python code will live.

We will run the “virtualenv” tool (now that you have aliased it) on the directory you create for your project, which essentially does a copy of the Python environment from the master environment into the virtual environment and sets up a symbolic link to the master interpreter for the corresponding Python interpreter version you select for the environment.

The examples below assume you first need to make your virtual environment directory, and so includes a “mkdir” command before running the virtualization tool on it. If you already have the directory created, you can optionally change to it and then run the “virtualenv” tool from within.

To create a virtual environment containing the Python 2.7 interpreter and libraries:

> mkdir <directory_name>
> virtualenv2 <directory_name>

or

> mkdir <directory_name>
> cd <directory_name>
> virtualenv2 .

To create a virtual environment containing the Python 3.5 interpreter and libraries, just change the 2 for 3 in the above commands.

You can verify that the virtual environment has been created by listing the directory contents with ls -ll:

total 16
drwxr-xr-x 8 twillenberg staff 272B Sep 4 09:08 ./
drwxr-xr-x+ 51 twillenberg staff 1.7K Sep 4 08:03 ../
lrwxr-xr-x 1 twillenberg staff 63B Sep 4 07:57 .Python -> /System/Library/Frameworks/Python.framework/Versions/2.7/Python
drwxr-xr-x 16 twillenberg staff 544B Sep 4 07:57 bin/
-rw-r — r — 1 twillenberg staff 0B Sep 4 09:08 dir.text
drwxr-xr-x 3 twillenberg staff 102B Sep 4 07:57 include/
drwxr-xr-x 3 twillenberg staff 102B Sep 4 07:57 lib/
-rw-r — r — 1 twillenberg staff 60B Sep 4 07:57 pip-selfcheck.json

The first thing you will notice is some new directories. These directories contain the Python interpreter and standard libraries. There is also a symbolic link called .Python to the master Python interpreter which implies what version the virtual environment is.

Step 4: Activate a virtualised environment

The “activating” action makes a particular virtual environment the current one.

To “activate” the environment, making sure you are in the virtual environment directory, then issue the following commands:

> cd <directory_name>
> source ./bin/activate

You will notice that the name of the virtual environment now prepends the terminal prompt in parentheses.

Step 5: Test the virtual environment

To ensure that you are now inside your sandboxed virtual environment correctly, re-issue the which python or which python3 or which pip or which pip3 will now reveal new paths to the interpreters located in your virtual environment. If you run a “pip list” command, the modules loaded into the virtual environment will be listed. Doing a pip install <module_name> will result in an installation of modules into only the currently activated virtual environment.

Step 6: Deactivating a virtual environment

To switch off the virtual environment or “jump out”, type:

> deactivate

You will notice that the virtual environment name in parentheses has now disappeared from the terminal prompt.