Geek Culture
Published in

Geek Culture

Open Source / Python Community

Pydantic, Special Snowflakes, and Toxic Self-Importance

Throwing a fit isn’t collaboration.

My History with Python Typing

I’ve been a full-time Python developer for over 5 years. It was my first language and I love it. I made the transition over to Python 3.7 about 4 years ago and have been contributing to open-source since then. One of the things that excited me the most about Python’s type-system was the ability to describe and then enforce types at runtime in a way that’s not really possible with standard statically-typed language. Not long after I moved to 3.7 I released my own library for runtime type-analysis: Typical.

Not long after I released the first version of Typical, a coworker of mine showed me Pydantic. It cropped up on a similar timeline as my own library and naturally I was curious. The major difference between our libraries at that time was that my library was mainly concerned with enforcing function annotations, whereas Pydantic was (and still is) mainly focused on defining data models. It’s a batteries-included library that does nearly everything you need and more and I have great respect for the work they’ve put into it to try and make folk’s lives easier.

Evaluating Pydantic’s Implementation

I evaluated Pydantic and learned a lot from its implementation. The number one thing I learned was to never approach building a library in the way Pydantic has done. I’ll readily admit this is generally an esoteric issue and a matter of personal taste, but I do contend it leads to real issues that create constant, ongoing pain in order to support.

The problem with Pydantic’s implementation of runtime type analysis and casting is that it attempts to essentially build its own type system in parallel to Python’s. That starts with how classes themselves are actually built. By going all-in on metaclasses, Pydantic is hijacking the natural class construction defined by Python itself. This means a few things:

  1. If you want to use Pydantic, you must use it everywhere and you must adopt its type-system (e.g., everything must be a Pydantic BaseModel).
  2. As with many Python ORMs, the “classes” you define are fundamentally different in behavior and upon inspection than a vanilla class or even a dataclass. (Keyword-only parameters, no defined signature, custom plugins for IDE or static type analysis, etc.)
  3. Pydantic tries to emulate popular interfaces (e.g., dataclasses) but it doesn’t really work with Pydantic’s type system, leading to numerous issues.

Esoteric reasons aside, I’ve acknowledged that Pydantic is popular and it’s helping people do their jobs, so I haven’t really had a beef with the library, I just don’t personally care for the implementation.

Toxic Self-Importance Collides with Questionable Implementation

My attitude changed a few days ago, when the author of Pydantic submitted an issue regarding the now-delayed feature of Postponed Evaluation of Annotations (PEP 563). In Issue 2678, Pydantic’s maintainer makes the bold assertion that the changes introduced in PEP 563 will fundamentally break the entire ecosystem of runtime type analysis.

This is a false assertion. I’ve done an audit of the related issues in Pydantic and found only one case my library didn’t support, which was resolved with about 40 lines of code I wrote over the period of about 2 hours. I’ve been using PEP 563 annotations in my own code for over a year now and have made a few changes to my own library to ensure support moving forward. In general, postponed evaluation has vastly improved my ability to support complex use-cases, such as recursive or self-referencing types.

Pydantic waited until days before the beta release of Python 3.10 to open an issue (on its own issue board!) as a call-to-action for its community. Since the PEP was introduced and accepted, Pydantic has worked to support it, but never reached out to the SC or the authors of the PEP to communicate its concerns and work with them for a resolution. If it had concerns with PEP 563 and Python 3.10, it should have been proactive in its communications to the SC for the last 2 years rather than throw a public fit and force the SC to completely alter course in the days running up to the release deadline.

This is not how collaboration works. This is how that self-important dev you hate to work with always gets their way. Jumping on the bandwagon encourages this toxic (and too often tolerated) behavior.

The Tail Wagging the Dog

Unfortunately, the conversation is now dominated by Pydantic’s complaints and the years of issues in Pydantic’s backlog related to supporting PEP 563 look like absolute proof that runtime analysis is severely crippled by this feature when the reality is very different. I don’t take issue with CPython’s core devs and the SC for choosing to delay this feature — it was really their only choice. But Pydantic made a deliberate choice to hit the panic button and start a fire rather than collaborate and work with the very people its library rely upon. This Reddit user sums it up nicely:

The first thing that came to mind when seeing this is

Did anyone from pydantic/fast api ever communicate with the core devs about this?

This isn’t an issue that popped up out of nowhere. It’s been known about in pydantic since 2018 and numerous tickets have been filed and worked on, but seems like a whack-a-mole recurring issue that can’t get solved satisfactorily.

Colvin is essentially acting like a manager that is trying to block a major release in a very last minute fashion because it breaks their project in ways they’ve known about for years shortly before the release is feature frozen. Come on. We’ve all had to deal with that from management, so why are we inflicting this on ourselves??

I hope we can move forward from this without severely hampering the ability of Python to advance. We now have a precedent of which we need to be extremely wary.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store