Improve Your Python Package Management With pipenv
There are better ways to manage your packages and virtual environments than using pip and virtualenv
A while ago, I wrote an article on why you should be using virtual environments instead of installing your packages globally. The point of the article is valid: Stop installing packages globally.
After explaining why you should really be using virtual environments, I started telling everyone to use
pip. And that’s where I went wrong.
Many developers still use these tools, but most of them probably haven’t heard about
pipenv yet. This article serves to rectify my mistake!
pipenv has a number of advantages compared to using
virtualenv. These are the main ones:
- You no longer need to use
virtualenvseparately. Instead, you have one tool that does it all — and more!
pipenvseparates your top-level dependencies from the last tested combination (e.g. the output of
pip freeze). This makes dependency management more user-friendly for you as a developer.
pipenvencourages the use of the latest versions of dependencies to minimize security risks. It can even scan your dependencies for known vulnerabilities.
pipenvgives insight into your dependency graph with
pipenvhashes all dependencies. It will detect packages that have been tampered with after you initially included it as a dependency.
pipenvcan work with
requirements.txtfiles too. If there is one, it will automatically detect it and convert it into a Pipfile.
$ brew install pipenv
If you’re on Linux, you should check out Homebrew as well since they ported it to Linux too.
Alternatively, chances are you have
pip installed as a system-wide package. I do this on my systems. Sure, I use virtual environments for all my projects. But I also installed a couple of tools like
pip as system-wide packages. If you have
pip installed, simply use it to install
$ pip install --user pipenv
--user option installs
pipenv for the local user only. If you use it, you ensure that you won’t interfere with system-wide packages. Honestly, I just installed it globally without any problems.
Now that you have
pipenv installed, let’s try it!
First, create a directory. I called mine
myenv. Now cd into the directory and install your first dependency with
$ pipenv install requests
What happens next is:
pipenvwill detect there’s no virtual environment yet, so it will create one.
- It will install requests.
- It creates two files:
The following screenshot shows
pipenv in action:
After it’s done, you can optionally enter the virtual environment with:
$ pipenv shell
This is optional because you can also launch a command in the virtual environment like this:
$ pipenv run <your command>
Where’s My virtualenv?
I don’t like black boxes. I want to know what’s going on. So let’s first inspect
Pipfile. It should look like this:
As you can see, Pipfile contains our top-level requirements without specifying a version number because we didn’t specify it. Now take a look at
Pipfile.lock. It’s a big file, so I’ll just show a snippet here to illustrate:
Requests and all of its dependencies are listed, including the version numbers. Basically, this is an advanced version of the output of
pip freeze. These are the versions we worked with. It’s the combination of packages that are guaranteed to work.
But where’s the virtual environment? If you look closely at the screenshot above, you see the
virtualenv is created in
~/.local/share/virtualenvs/myenv-mAvwj65b/. We could have asked
pipenv too with
If you’re familiar with the structure of a virtual environment, you’ll notice that this one is no different. It’s just located somewhere else, outside of your project structure. This way, you don’t need to exclude it from your version control system too.
If you create another virtual environment with the same name in another location, it won’t be reused. The hash in the directory name is, I presume, based on the path to your
pipenv uses it to find the correct one despite double names.
Separating Development Packages
With pip, you are only able to separate development requirements from production requirements by using a second requirements file (e.g.
pipenv, you can simply use the
$ pipenv install pytest --dev
Another developer that starts working on your project can install all requirements, including the developer requirements, using:
$ pipenv install --dev
pipenv attempts to install all sub-dependencies required by your core dependencies. Conflicts might arise, though. If package A requires at least version 1.10 of package B, while package C requires version 1.9, you have a problem.
pipenv can’t solve it for you, but it will warn you about it and refuse to continue by default.
pipenv also offers nice insight into your dependencies with the
Detection of Security Vulnerabilities
pipenv can scan your packages for security vulnerabilities. It does so by using the safety package. Enter:
$ pipenv check
pipenv will scan your dependency graph for known security vulnerabilities!
Alternatives to pipenv
pip + virtualenv
Many projects are based on these two tools simply because they've been around much longer. It’s good to at least know about them and know their basic usage, if only because
pipenv is built on top of these.
Anaconda offers similar functionality to
pipenv, but it’s from a commercial vendor. From their own documentation, “Anaconda® is a package manager, an environment manager, a Python/R data science distribution, and a collection of over 7,500+ open-source packages.”
So, it does a lot and has its place in the scientific and data science industries. It’s a bit too heavy for me, though. I like
Thank you for reading. If you notice mistakes, please let me know. You’ll be helping lots of people.