Stop Installing Python Packages Globally — Use Virtual Environments

With virtual environments, you can isolate your software and avoid version conflicts

Erik-Jan van Baaren
Feb 4 · 5 min read
Photo by Ryan Loughlin on Unsplash

Python virtual environments allow you to install Python packages in an isolated location for a particular application, instead of installing them globally.

Let’s explore what the advantages are and how you can quickly get started.


Why You Need Virtual Environments

If you install third-party packages system-wide, you could argue that you’re being very efficient. After all, you only need to install it once, saving you precious time and disk space.

There’s a problem with this approach that may start to unfold weeks or months later, however.

Suppose your project, Project A, is written against a specific version of its dependencies. In the future, you might need to upgrade one of these dependencies.

Perhaps because you need the latest version for another project you started, called Project B. Or, you might need an ancient version of a dependency to get a piece of old and unmaintained code running in a hurry.

Once you did this, however, your Project A’s code may break badly, since APIs can be completely different between major versions of a dependency.

A virtual environment fixes this problem by isolating your project from other projects and system-wide packages.

A virtual environment also helps other users of your software. In conjunction with pip and a requirements.txt file, a virtual environment will help others to reproduce the exact environment for which your software was built.

If you’re working on a shared host, like those at a university or a web hosting provider, you won’t be able to install packages globally since you don’t have the administrator rights to do so.

In these places, a virtual environment allows you to install anything you want locally.


Python Virtual Environments vs. Other Options

There are several options to isolate your project from others. In the most extreme case, you could buy a second PC and run your code there. Problem fixed! It’s a bit expensive though…

A virtual machine is a much cheaper option but still requires you to install a complete operating system. A bit of a waste as well, right?

Next in line is containerization, with the likes of Docker and Kubernetes. These can be very powerful, but for a small Python project still a bit too much and too complicated.

And that’s where venv comes in, Python’s ultra-lightweight virtual environment.


Create a venv

Enough theory! Let’s get our hands dirty. There are several ways to create a Python virtual environment, depending on the Python version you are running.

But before you read on, I’d like to direct you to a newer article of mine about pipenv. It’s a tool that combines the functionality of virtualenv and pip:

There’s nothing wrong about using virtualenv and pip. Many individuals and teams use it in their daily work. But pipenv does offer a number of advantages that you should at least read about before you decide on what to use.

If you are running Python 3.4+, you can use the venv module baked into Python:

$ python -m venv [directory]

This command will create a venv in the specified directory and copy pip and easy_install into it too.

The alternative that works for any Python version is using the virtualenv package. You may need to install it first, system-wide, with:

$ sudo pip install virtualenv

Once installed, you can create a virtual environment with:

$ virtualenv [directory]

Activate the Virtual Environment

We activate our virtual environment with the source command. If you created your venv in the myvenv directory, the command would be:

$ source myvenv/bin/activate

That’s it! We’re ready to rock!


Install Packages

We can now use pip to install packages inside our newly created environment. Pip is installed inside your virtual environment by default. Because everything is installed locally, you don’t need to use sudo.

To install simplejson, simply use:

$ pip install simplejson

And to upgrade pip to the latest version, you can use pip too since it’s just another package inside your venv:

$ pip install --upgrade pip

In a virtual environment, it’s a good habit to install specific versions of packages. It ensures that you reap the full benefits of using virtual environments in the first place.

After all, we do this to make sure our software always works as intended, by fixating to specific dependency versions.

A requirements.txt file contains a very simple list of dependencies, one per line. In its most simple form, it could look like this:

simplejson
chardet

But what we really want is to fixate the versions. That’s not hard either:

chardet==3.0.4
simplejson==3.17.0

You can also relax these constraints a little, by using >= and <=, or even a combination of those:

chardet>=3.0.0,<=3.1.0
simplejson>=3.17.0

You can make your life a little easier by creating your requirements file using pip’s freeze option.

First, you write your software and install all the requirements you need as you go with pip. Once you’re done, simply use the following command:

$ pip freeze > requirements.txt

Pip will create a requirements file with all the currently installed dependencies, including version numbers. Neat!

Finally, to install all the dependencies listed in this file, use:

pip install -r requirements.txt

How It Works

When you activate a virtual environment, your $PATH variable is changed. You can see it for yourself, by printing the path with echo $PATH. In my case, it looks like this:

/Users/ej/myvenv/bin:/usr/local/bin:/usr/bin:/bin:etcetera

As you can see, the bin directory of your venv is put in front of everything else, overriding all the system-wide Python software.

If you take a look inside the virtual environment directory of your venv, you’ll see something like this:

.
├── bin
│ ├── activate
│ ├── activate.csh
│ ├── activate.fish
│ ├── easy_install
│ ├── easy_install-3.7
│ ├── pip
│ ├── pip3
│ ├── pip3.7
│ ├── python -> python3
│ └── python3 -> /usr/local/bin/python3
├── include
├── lib
│ └── python3.7
│ └── site-packages
└── pyvenv.cfg

As you can see, venv also fixes the Python version to the version with which you created the venv, by creating a symlink to it.

Any packages you install end up in the site-packages directory.


Deactivate a Virtual Environment

If you are done working on your project, it’s a good habit to deactivate its venv. Without deactivating it, all other Python code you execute will also run inside it.

Deactivating your virtual environment couldn’t be simpler. Just enter this:

$ deactivate

Wrap up

With these simple steps, you’ll get up and running with venv:

  1. python -m venv venv
  2. source venv/bin/activate
  3. Create a requirements.txt file with your required packages.
  4. pip install -r requirements.txt
  5. When you’re finished, deactivate it with deactivate.

Better Programming

Advice for programmers.

Erik-Jan van Baaren

Written by

A writer at heart and software/data engineer by profession. Subscribe to my low-volume newsletter at https://techexp.substack.com/

Better Programming

Advice for programmers.

More From Medium

More from Better Programming

More from Better Programming

More from Better Programming

The Zero-Dollar Infrastructure Stack

1.1K

More from Better Programming

More from Better Programming

Fun Side Projects That You Can Build Today

3.1K

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade