Improve Your Python Package Management With pipenv

There are better ways to manage your packages and virtual environments than using pip and virtualenv

Erik-Jan van Baaren
Mar 10 · 5 min read
Photo by Alvaro Reyes on Unsplash.

A while ago, I wrote an article on why you should be using virtual environments instead of installing your packages globally. The point of the article is valid: Stop installing packages globally.

After explaining why you should really be using virtual environments, I started telling everyone to use virtualenv and pip. And that’s where I went wrong.

Many developers still use these tools, but most of them probably haven’t heard about pipenv yet. This article serves to rectify my mistake!


Why pipenv?

Using pipenv has a number of advantages compared to using pip and virtualenv. These are the main ones:

  • You no longer need to use pip and virtualenv separately. Instead, you have one tool that does it all — and more!
  • pipenv separates your top-level dependencies from the last tested combination (e.g. the output of pip freeze). This makes dependency management more user-friendly for you as a developer.
  • pipenv encourages the use of the latest versions of dependencies to minimize security risks. It can even scan your dependencies for known vulnerabilities.
  • pipenv gives insight into your dependency graph with pipenv graph.
  • pipenv hashes all dependencies. It will detect packages that have been tampered with after you initially included it as a dependency.
  • pipenv can work with requirements.txt files too. If there is one, it will automatically detect it and convert it into a Pipfile.

Installing pipenv

On MacOS, I strongly recommend Homebrew. If you are not using Homebrew already, please read my article on it and install it. After that, it’s as simple as:

$ brew install pipenv

If you’re on Linux, you should check out Homebrew as well since they ported it to Linux too.

Alternatively, chances are you have pip installed as a system-wide package. I do this on my systems. Sure, I use virtual environments for all my projects. But I also installed a couple of tools like pip as system-wide packages. If you have pip installed, simply use it to install pipenv:

$ pip install --user pipenv

The --user option installs pipenv for the local user only. If you use it, you ensure that you won’t interfere with system-wide packages. Honestly, I just installed it globally without any problems.


Using pipenv

Now that you have pipenv installed, let’s try it!

First, create a directory. I called mine myenv. Now cd into the directory and install your first dependency with pipenv:

$ pipenv install requests

What happens next is:

  • pipenv will detect there’s no virtual environment yet, so it will create one.
  • It will install requests.
  • It creates two files: Pipfile and Pipfile.lock.

The following screenshot shows pipenv in action:

pipenv in action. Screenshot by author.

After it’s done, you can optionally enter the virtual environment with:

$ pipenv shell

This is optional because you can also launch a command in the virtual environment like this:

$ pipenv run <your command>

Where’s My virtualenv?

I don’t like black boxes. I want to know what’s going on. So let’s first inspect Pipfile. It should look like this:

As you can see, Pipfile contains our top-level requirements without specifying a version number because we didn’t specify it. Now take a look at Pipfile.lock. It’s a big file, so I’ll just show a snippet here to illustrate:

Requests and all of its dependencies are listed, including the version numbers. Basically, this is an advanced version of the output of pip freeze. These are the versions we worked with. It’s the combination of packages that are guaranteed to work.

But where’s the virtual environment? If you look closely at the screenshot above, you see the virtualenv is created in ~/.local/share/virtualenvs/myenv-mAvwj65b/. We could have asked pipenv too with pipenv --venv.

If you’re familiar with the structure of a virtual environment, you’ll notice that this one is no different. It’s just located somewhere else, outside of your project structure. This way, you don’t need to exclude it from your version control system too.

If you create another virtual environment with the same name in another location, it won’t be reused. The hash in the directory name is, I presume, based on the path to your virtualenv. pipenv uses it to find the correct one despite double names.


Separating Development Packages

With pip, you are only able to separate development requirements from production requirements by using a second requirements file (e.g. requirements-dev.txt). With pipenv, you can simply use the --dev option:

$ pipenv install pytest --dev

Another developer that starts working on your project can install all requirements, including the developer requirements, using:

$ pipenv install --dev

Dependency Management

pipenv attempts to install all sub-dependencies required by your core dependencies. Conflicts might arise, though. If package A requires at least version 1.10 of package B, while package C requires version 1.9, you have a problem. pipenv can’t solve it for you, but it will warn you about it and refuse to continue by default.

pipenv also offers nice insight into your dependencies with the graph command:


Detection of Security Vulnerabilities

pipenv can scan your packages for security vulnerabilities. It does so by using the safety package. Enter:

$ pipenv check

pipenv will scan your dependency graph for known security vulnerabilities!


Alternatives to pipenv

pip + virtualenv

Many projects are based on these two tools simply because they've been around much longer. It’s good to at least know about them and know their basic usage, if only because pipenv is built on top of these.

Anaconda

Anaconda offers similar functionality to pipenv, but it’s from a commercial vendor. From their own documentation, “Anaconda® is a package manager, an environment manager, a Python/R data science distribution, and a collection of over 7,500+ open-source packages.”

So, it does a lot and has its place in the scientific and data science industries. It’s a bit too heavy for me, though. I like pipenv more!


Conclusion

Thank you for reading. If you notice mistakes, please let me know. You’ll be helping lots of people.

Better Programming

Advice for programmers.

Thanks to Zack Shapiro

Erik-Jan van Baaren

Written by

A writer at heart and software/data engineer by profession. Subscribe to my low-volume newsletter at https://techexp.substack.com/

Better Programming

Advice for programmers.

More From Medium

More from Better Programming

More from Better Programming

More from Better Programming

Fun Side Projects That You Can Build Today

3K

More from Better Programming

More from Better Programming

The Zero-Dollar Infrastructure Stack

1.1K

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade