Best practices for Python projects using Virtual Environment

img src: https://res-1.cloudinary.com/hackers/image/upload/q_auto:best/v1/2019/11/virtualenv.jpg

When managing a Python project, there comes a time to keep Python packages separated from your main environment. There may be incompatible versions that irritate each other and cause bizarre errors.

To avoid these things, you need to create virtual environments, and historically, virtualenv has been just the tool to do this in the Python universe.

A virtual environment is a tool that helps to keep dependencies required by different projects separate by creating isolated Python virtual environments for them.

When and where to use a virtual environment?

Let's assume that we are working on two python projects using some package ‘X’ (X could be Django, flask, etc). One of the projects uses X of version V1 and the other project uses X of version V2. In this situation, the virtual environment can be really useful to maintain the dependencies of both the projects. By default, every project on your system will use these same directories to store and retrieve site-packages (third party libraries). How does this matter? Now, in the above example of two projects, you have two versions of X. This is a real problem for Python since it can’t differentiate between versions in the “site-packages” directory. So both V1 and V2 would reside in the same directory with the same name. This is where virtual environments come into play. To solve this problem, we just need to create two separate virtual environments for both the projects. The great thing about this is that there are no limits to the number of environments you can have since they’re just directories containing a few scripts.

Virtual Environment should be used whenever you work on any Python-based project. It is generally good to have one new virtual environment for every Python-based project you work on. So the dependencies of every project are isolated from the system and each other.

How does a virtual environment work?

Module named virtualenv which is a tool used to create isolated Python environments. virtualenv creates a folder that contains all the necessary executables to use the packages that a Python project would need.

Guide for VirtualEnv

The tool virtualenv runs on both Python2 and Python3.

Installation of Virtualenv

$ pip install virtualenv

Creating and Initiating the Virtual Environment

$ virtualenv name     // created virtual Environment 'name'

After running this command, a directory named ‘name’ will be created. This is the directory that contains all the necessary executables to use the packages that a Python project would need. This is where Python packages will be installed.
If you want to specify Python interpreter of your choice, for example, Python 3, it can be done using the following command

$ virtualenv -p /usr/bin/python3 name

To create a Python 2.7 virtual environment, use the following command

$ virtualenv -p /usr/bin/python2.7 name

After creating the virtual environment, you need to activate it. Remember to activate the relevant virtual environment every time you work on the project.

$ source name/bin/activate  # for bash/zsh shell
$ . name/bin/activate.fish  # for fish shell
C:/> name\Scripts\Activate.ps1 # for Windows cmd.exe

Once the virtual environment is activated, the name of your virtual environment will appear on the left side of the terminal. This will let you know that the virtual environment is currently active. Now we can install dependencies related to the project in this virtual environment.

$ (name)$ pip install flask

The Flask package will be placed in ‘name’ folder and will be isolated from the complete system.

We can see the packages installed in this virtual environment using

$ pip list
Package      Version
------------ -------
Click        7.0
Flask        1.0.2
itsdangerous 1.1.0
Jinja2       2.10
MarkupSafe   1.1.0
pip          18.1
setuptools   40.6.2
Werkzeug     0.14.1
wheel        0.32.3

Deactivating the Virtual Environment

Once you are done with the work, you can deactivate the virtual environment by the following command

$ (name)$ deactivate

After leaving the virtual environment, you can see that packages in our main environment are quite different.

$ pip list
pip (8.1.1)
pycrypto (2.6.1)
setuptools (20.7.0)
virtualenv (16.1.0)
wheel (0.29.0)

Removing the Virtual Environment

$ rm -r /path/to/name

Few reference commands of virtualenv

virtualenv [OPTIONS] ENV_DIR

Where ENV_DIR is an absolute or relative path to a directory to create the virtual environment in.

Options

--version show program’s version number and exit

-h, --help show this help message and exit

-v, --verbose Increase verbosity.

-q, --quiet Decrease verbosity.

--clear Clear out the non-root install and start from scratch.

--system-site-packages Give the virtual environment access to the global site-packages.

--always-copy Always copy files rather than symlinking.

--relocatable Make an EXISTING virtualenv environment relocatable. This fixes up scripts and makes all .pth files relative.

--unzip-setuptools Unzip Setuptools when installing it.

--extra-search-dir=DIR Directory to look for setuptools/pip distributions in. This option can be specified multiple times.

References

Reference Guide - virtualenv 16.7.7 documentation

venv - Creation of virtual environments - Python 3.8.0 documentation