Stop Installing Python Packages Globally — Use Virtual Environments
With virtual environments, you can isolate your software and avoid version conflicts
Python virtual environments allow you to install Python packages in an isolated location for a particular application, instead of installing them globally.
Let’s explore what the advantages are and how you can quickly get started.
Why You Need Virtual Environments
Preventing version conflicts
If you install third-party packages system-wide, you could argue that you’re being very efficient. After all, you only need to install it once, saving you precious time and disk space.
There’s a problem with this approach that may start to unfold weeks or months later, however.
Suppose your project, Project A, is written against a specific version of its dependencies. In the future, you might need to upgrade one of these dependencies.
Perhaps because you need the latest version for another project you started, called Project B. Or, you might need an ancient version of a dependency to get a piece of old and unmaintained code running in a hurry.
Once you did this, however, your Project A’s code may break badly, since APIs can be completely different between major versions of a dependency.
A virtual environment fixes this problem by isolating your project from other projects and system-wide packages.
Easy to reproduce
A virtual environment also helps other users of your software. In conjunction with pip and a
requirements.txt file, a virtual environment will help others to reproduce the exact environment for which your software was built.
Works everywhere, even when not root
If you’re working on a shared host, like those at a university or a web hosting provider, you won’t be able to install packages globally since you don’t have the administrator rights to do so.
In these places, a virtual environment allows you to install anything you want locally.
Python Virtual Environments vs. Other Options
There are several options to isolate your project from others. In the most extreme case, you could buy a second PC and run your code there. Problem fixed! It’s a bit expensive though…
A virtual machine is a much cheaper option but still requires you to install a complete operating system. A bit of a waste as well, right?
Next in line is containerization, with the likes of Docker and Kubernetes. These can be very powerful, but for a small Python project still a bit too much and too complicated.
And that’s where venv comes in, Python’s ultra-lightweight virtual environment.
Create a venv
Enough theory! Let’s get our hands dirty. There are several ways to create a Python virtual environment, depending on the Python version you are running.
But before you read on, I’d like to direct you to a newer article of mine about
pipenv. It’s a tool that combines the functionality of
Improve Your Python Package Management With pipenv
There are better ways to manage your packages and virtual environments than using pip and virtualenv
There’s nothing wrong about using
pip. Many individuals and teams use it in their daily work. But
pipenv does offer a number of advantages that you should at least read about before you decide on what to use.
Python 3.4 and above
If you are running Python 3.4+, you can use the
venv module baked into Python:
$ python -m venv [directory]
This command will create a
venv in the specified directory and copy
easy_install into it too.
All other versions
The alternative that works for any Python version is using the
virtualenv package. You may need to install it first, system-wide, with:
$ sudo pip install virtualenv
Once installed, you can create a virtual environment with:
$ virtualenv [directory]
Activate the Virtual Environment
We activate our virtual environment with the
source command. If you created your venv in the
myvenv directory, the command would be:
$ source myvenv/bin/activate
That’s it! We’re ready to rock!
We can now use pip to install packages inside our newly created environment. Pip is installed inside your virtual environment by default. Because everything is installed locally, you don’t need to use
simplejson, simply use:
$ pip install simplejson
And to upgrade pip to the latest version, you can use pip too since it’s just another package inside your venv:
$ pip install --upgrade pip
Using a requirements.txt file
In a virtual environment, it’s a good habit to install specific versions of packages. It ensures that you reap the full benefits of using virtual environments in the first place.
After all, we do this to make sure our software always works as intended, by fixating to specific dependency versions.
requirements.txt file contains a very simple list of dependencies, one per line. In its most simple form, it could look like this:
But what we really want is to fixate the versions. That’s not hard either:
You can also relax these constraints a little, by using
<=, or even a combination of those:
You can make your life a little easier by creating your requirements file using pip’s freeze option.
First, you write your software and install all the requirements you need as you go with pip. Once you’re done, simply use the following command:
$ pip freeze > requirements.txt
Pip will create a requirements file with all the currently installed dependencies, including version numbers. Neat!
Finally, to install all the dependencies listed in this file, use:
pip install -r requirements.txt
How It Works
When you activate a virtual environment, your
$PATH variable is changed. You can see it for yourself, by printing the path with
echo $PATH. In my case, it looks like this:
As you can see, the bin directory of your venv is put in front of everything else, overriding all the system-wide Python software.
If you take a look inside the virtual environment directory of your venv, you’ll see something like this:
│ ├── activate
│ ├── activate.csh
│ ├── activate.fish
│ ├── easy_install
│ ├── easy_install-3.7
│ ├── pip
│ ├── pip3
│ ├── pip3.7
│ ├── python -> python3
│ └── python3 -> /usr/local/bin/python3
│ └── python3.7
│ └── site-packages
As you can see, venv also fixes the Python version to the version with which you created the venv, by creating a symlink to it.
Any packages you install end up in the
Deactivate a Virtual Environment
If you are done working on your project, it’s a good habit to deactivate its venv. Without deactivating it, all other Python code you execute will also run inside it.
Deactivating your virtual environment couldn’t be simpler. Just enter this:
With these simple steps, you’ll get up and running with venv:
python -m venv venv
- Create a
requirements.txtfile with your required packages.
pip install -r requirements.txt
- When you’re finished, deactivate it with