A Virtual World for Your Project

Creating Python Virtual Environments

Why do you need it?

When you start learning python or any program, you’ll mostly try to install the latest version and packages. But imagine a scenario where you are working on two different web-based python projects: one of them is built on Flask 0.12, and the other is built on Flask 1.0. Both will have different dependencies and you may not be able to upgrade the packages without any issue.

In a system with a larger number of dependencies and in a rapidly developing field, this could be the worst nightmare. In such situations, the virtual environment can be really useful to maintain separate dependencies for both the projects.

Aside from this situation, there are many other use cases when having additional environments is really useful. You might have faced a common problem, that a code that once worked beautifully does not work when you try to run it today.

Perhaps, one of the packages is no longer compatible with the other parts of your program, popularly termed as ‘the breaking changes’.

If you had set up an environment for your application, that contained the compatible Python version and the packages, you might have not faced such issues.

Setting up an environment will be extremely helpful when delivering your application to your client or when collaborating with your team member. This will make sure that your application runs smoothly on their computers.

How does it work?

File Structure of Python Virtual Environment Folder

By now, you might have the understanding that the virtual environment is probably one of the most important tools for Python developers to keep project dependencies isolated. But how does it work?

The virtual environment tool creates a folder inside the project directory or in another dedicated location. When the virtual environment is activated, you will install the packages inside this project-specific virtual environment folder using pip or condo. The virtual environment folders are package management system used to install and manage software packages written in Python.

Syntax: pip install packagename.

Here’s what the folder contains:

• bin: files that interact with the virtual environment. We will source the activate file to enter in the virtual environment.
• include: C headers that compile the Python packages.
• lib: a copy of the Python version along with a site-packages folder, where each dependency is installed.

Furthermore, there are shortcuts or symlinks to a few different Python tools and to the Python executables themselves. These files are used to ensure that all Python code and commands are executed within the context of the current environment, which is how the isolation from the global environment is achieved.

Which Virtual Environment Tool to Use?

If you have an IDE (Integrated Development Environment) such as PyCharm, Eclipse or others you can create it when setting up your project, generally while selecting the Python interpreter.

Standard library:

venv is a lightweight virtual environment tool package shipped with Python 3, which you can run using $ python3 -m venv /pathtovirtualenvironment.

Each virtual environment has its own Python binary (which matches the version of the binary that was used to create this environment) and can have its own independent set of installed Python packages in its site directories.

PyPI(Python Package Index) packages not in the standard library:

virtualenv is a very popular tool to create isolated Python environments for Python libraries. It works by installing a bunch of files in a directory (eg: env/) and then modifying the PATH environment variable to prefix it with a custom bin directory (eg: env/bin/).

An exact copy of the python or python3 binary is placed in this directory. But, Python is programmed to look for libraries in the environment directory, relative to its path first.
$ virtualenv -p /usr/bin/python3 virtualenv_name

pyenv allows us to isolate different Python versions and switch between them. These are special scripts that decide on the fly which version of Python to run based on the PYENV_VERSION environment variable, the .python-version file, and the ~/.pyenv/version file.

virtualenvwrapper is a set of extensions to virtualenv (see docs). It gives you commands like mkvirtualenv, lssitepackages, and workon for switching between different virtualenv directories. This tool is especially useful if you want multiple virtualenv directories.

Pipenv is a recent project which aims to combine Pipfile, pip and virtualenv into one command on the command-line. The virtualenv directory typically gets placed inside ~/.local/share/virtualenvs/. This is different from virtualenv, where the directory is typically in the current working directory.

Conda isn’t just a virtual environment. It is an open source package management, dependency management, and environment management for any language — Python, R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, FORTRAN.

You can use conda as a package manager to automate the process of installing, updating, and removing packages. Conda can be combined with continuous integration systems such as Travis CI and AppVeyor to provide frequent, automated testing of your code.

There are different distribution you can use for conda:

• Anaconda comes with a GUI and numerous packages bundled together and takes more than 3GB of disk space.
• Miniconda is smaller and requires about 400MB disk space, and it contains only a few basic packages. Generally preferred.

A Short Tutorial to Getting Started with Conda

Here’s a short tutorial on using a conda but you could also use virtualenv or python3 venv along with pip in a similar manner.

List all available conda env on your machine using
conda info --env or
conda env list

Create a new environment named myenv
conda create — name myenv python=3.4

Activate your environment
activate myenv for Windows
source activate myenv or conda activate myenvfor Linux & Mac

Installing packages in the activate env (use keyword remove instead of install to remove)
conda install seaborn=0.7.0

You can also install PIP packages inside a Conda environment.
Updating Packages in the active env,
conda update seaborn

Ignoring package name updates all the installed packages. But if you don’t want to deal with compatibility issues (breaking changes) caused by a new version of one of the packages you use, you can prevent that package from updating by creating a file named pinned in the environment’s conda-metadirectory and add the list of those packages in the file.

You can verify the packages available in your env with the following command conda list -n myenv

If you want a different env then you can clone base using the following command
conda create --name myenv --clone base

And of course, you would want to deactivate from your environment
deactivate or conda deactivate