Pipenv review, after using it in production

If you are not subscribed to /r/Python on reddit, you’ve probably missed this thread discussing the current state of Python Packaging Authority, pipenv and poetry. It wouldn’t be all that interesting, if it wasn’t for the involvment of both authors of pipenv and poetry and PyPA representative. If you don’t mind looking through piles of emotions to find some interesting points, go ahead and read it.

The current state of python’s packaging is awful, I don’t think there’s anyone who could disagree with that. This problem is recognized and there are many attempts at trying to solve the mess. Pipenv was the first and it has gained a lot of traction, however it doesn’t sit well with everyone. And it’s also not suited for every project — like libraries.

I’ve been using Pipenv for quite a while now and I managed to introduce it to our team’s codebase. We’ve been working with Pipenv since around November 2017, I personally created my first issue on it’s GitHub back in May 2017 and here’s my experience:

The multiple environment problem

As issue number #368 I first started discussing multiple environments (e.g. Python 2.7 next to 3.6 for tests). I resolved my use case, but the issue is still being somewhat actively discussed through issue #1050. The tl;dr is — supporting multiple environments goes against Pipenv’s (therefore also Pipfile’s) philosophy of deterministic reproducible application environments. So if you want to use Pipenvfor a library, you’re out of luck. That means many projects just can not use Pipenv for their dependency managment.

The longer version shows Kenneth Reitz (an amazing part of the Python community and my personal inspiration) being stark and not very open to the community’s needs . I recommend reading the issue #1050 completely, to make an opinion for yourself. While his approach can be understood, it also helps understand why some people look at Pipenv with poor taste in their mouth. As much as I hate to, there’s no way to review Pipenv without commenting on it’s author and I apologize in advance, I’ll mention Kenneth again.

The stability problem

Unfortunately more than once, I came to work to see failed tests, that ran just fine the day before. And it was almost always Pipenv causing the issues. While the team was almost always very quick to fix the issues, if we were to encounter a critical bug in production and to deploy a hotfix, our CI’s would be failing and we’d have lost time identifying the issue and having to lock Pipenv to a previous version hoping it would work then.

If those were honest mistakes in a proper release process, I wouldn’t be too harsh, however it’s the release process that’s at fault here. Because there is none. Kenneth (here goes the second mention) fairly often goes on commit streaks in master branch, releasing along the way. We’ve all been there, having ten commits in a row fixing or reverting previous changes. However, not many of us are maintaining a GitHub project with almost 11 000 stars, a project with 170 000 downloads per month, a project officially recommended by python.org. That is quite a few people relying on the tool and as we all know: “With great power…”.

Next time another problem like this occurs, I’ll just pin Pipenv to a last working version, which I should have maybe done from the start, and leave it until I’m forced to update.

The good

• Pipfile and Pipfile.lock really are superior to requirements.txt. By a ton.
• While I disliked it at first, having flake8 and security check builtin in a single tool is great
• Installing (exclusively) from a private respository works very well. Instead of replacing a dotfile somewhere in the system, specifying the repository in Pipfile is great
• Creating a new Pipfile is very easy
• No problems introducing Pipenv to it’s new users
• No problems installing from a mixture of indexes, git repositores…
• With --sequential it is actually deterministic, as advertised
• Virtualenv is much easier to get into and understand
• Dependencies can be installed into system (e.g. in Docker) — our case.
• At no point did anyone in the team even mentioned getting rid of Pipenv — which is a lot better than it sounds

Conclusion

Pipenv is not perfect and I don’t know if it’ll ever be, but I’ve been happy with it and I do recommend it to other teams. However, I don’t think pipenv is the goal.

We as a community need a tool that works for both libraries and applications. Something that Pipenv specifically does not want to be. The last thing the python community needs is choice (2vs3). As Zen of Python says: “There should be one — and preferably only one — obvious way to do it.”. If we end up with multiple tools for multiple purposes, people will find a way to abuse all of them to do all of it anyway and things won’t be pretty.

… what about Poetry?

I wish I knew. I haven’t had much time to experiment since I started working on our team’s current (GDPR related) project. It does look exciting though!

Edit:

A very interesting and mostly constructive discussion was born under this article on reddit (here). I’ll edit in the “best of”, but since I’m human, therefore imperfect, I’d recommend you to go ahead and read it yourself :).

• Some applications are intended to run under multiple python environments too, like CLI apps, GUI-based apps and others. I am a backend developer, therefore my view on Application vs Library is flawed by that perspective. Those applications are out of luck as much as libraries when it comes to using pipenv. (credit to kurashu89, kihashi and dirn)
• setuptools and the rest of the already existing tools can already do pretty much everything the tools that will come after it. What the previous tools do not excel at is frendliness, barrier of entry, knowledge barrier.
• Pipenv has a lot more flaws than I highlighted in this review, but I didn’t list them because they didn’t bother me as much. Mainly because many of those are not set in stone, or are being worked on actively.