Python, 33 years later

a laptop screen, showing syntax-highlighted python code

Python was first released to the public in February 1991, back when the most powerful PC had a 50MHz CPU, featured a few MB of RAM, Windows was at version 3.0 and the Linux kernel wouldn’t even exist for another six months.

Out of all the languages created in that same period and with the notable exception of the blessed, internet-scripting-seal-of-approval-receiving JavaScript (and admittedly Java), Python remains the most popular, by far, outshining the whole programming field in the latest TIOBE index of 2023 as it surfs the tsunami wave of AI development.

There are intrinsic reasons for that, but the survival of the language over a period of more than 30 years in an ultra competitive space and amidst ever-morphing hardware and software ecosystems should at the very minimum be taken as the testament of two things:

• Python is adaptable
• Python is expressive

As with all things though, success doesn’t always correlate to quality and it’s always interesting to keep an open mind and stay critical.

The Python language is a language that has provenly found a wide niche, first in the scientific community, then in scripting, the web space and more recently around AI tooling.

It has stood the test of time, as well as the test of diversity. Yet, as stated in the introduction, it is also a 33-year old duck-typed multi-paradigm language, one that has survived through some of its limitations and continues to thrive despite of them.

In this article, we’ll try to shed light on what exactly some of those limitations appear to be in the light of the broader programming ecosystem, hoping that it will help us gain a better understanding of where the language is coming from, which should in turn help us gauge the direction it’ll be taking in the next few years.

1. Python’s lack of first-class support for functional approaches and immutability

Core building bricks of the functional approach as second-class citizens

Whether people like it or not, tenets of functional programming have largely spread at the turn of the 2010’s in the software engineering world.

Ideas such as immutability, pure functions, composability and stronger typing are now commonplace (see Ruby, JavaScript, Java, C++, Clojure, Swift, Rust, Scala, Elixir to name a few), helping developers avoid whole classes of errors and bugs in the process.

In that regard, Python now appears to be lagging behind, as important concepts and building bricks that help make a language “functional-friendly” are still way harder to use than they should be. To give a few examples of what it means in practice for developers:

• map returns a map object , which you can conveniently iterate upon, but doesn’t preserve the type of its input in a way that is convenient, so developers often end up re-converting results to lists, dicts, sets or tuples
• iterating over dicts requires the use of .items() and a for loop
• reduce is not first-class and needs to be lifted from functools since Guido Van Rossum, Python’s ✌️benevolent dictator for life✌️ explained about twenty years ago that it shouldn’t be part of the standard library, because reduce is always clearer written as an accumulation list
• no find function currently exists in the standard library, forcing developers to rely on filter, which doesn’t short-circuit and returns a special iterable type, a filter object
• Python’s only standard library immutable collection is the tuple, which only provides index-based access
• the documentation about frozen dataclasses clearly states that true object immutability is not really achievable in python (due to the nature of the language) and we can at best “emulate immutability”, which most likely provides some security but not full security

Underpowered lambdas

Anonymous functions used as closures are a core construct often found in functional programming approaches. In Python, they’re often achieved through the use of lambda

The main limitation of the lambda construct in Python is their inability to contain multiple statements, making them suitable only for simple , one-line expressions that do not need to either break, continue, print or do anything remotely useful. For example:

add = lambda x, y: x + y

Contrast that with the more powerful and capable arrow functions in JavaScript

const names = ["Alice", "Bob", "Charlie", "David"];
const namesInCaps = names.map((name) => {
    console.log(`processing ${name}`);
    return name.toUpperCase();
});

or blocks in Ruby

numbers = [1, 2, 3, 4, 5]
squared_numbers = numbers.map do |num|
  puts "squaring number: #{num}"
  num * num
end

With those limitations in place, adopting a functional style in Python is of course not impossible but it still feels way harder than it should be, especially for people that dabble in other similar languages whose functional tooling is more refined.

2. Python’s limited support for first-class OOP

Limited support for proper encapsulation

Due to the open nature of the Python language, encapsulation is currently a matter of conventions, rather than something being enforced by the language itself. No support for visibility modifiers such as private , public or protected exists and this has to be tracked and managed manually through the use of naming conventions, such as using _ to denote that an attribute should not be visible outside of the class where it’s defined.

The fact that all attributes are visible to everyone can lead to situations where the state of objects ends up being mistakenly modified, in breach of the contract those objects have with the world. This can lead to bugs that are difficult to track down and debug, especially in larger codebases with multiple contributors.

Limited support for interfaces

Though Python supports multiple inheritance, it has no real native support for interfaces.

One way of to circumvent that limitation is through the use of naked abstract base classes (i.e. abstract base classes that provide no implementation at all)

from abc import ABC, abstractmethod
class Shape(ABC):
    @abstractmethod
    def area(self):
        pass
    
    @abstractmethod
    def perimeter(self, unit: str) -> int:
        pass

class Rectangle(Shape):
    def __init__(self, width, height):
        self.width = width
        self.height = height
        self.area = 3
    def perimeter(self) -> int:
        return 2 * (self.height + self.width)

This example showcases both the flexibility and the possible pitfalls of the feature: no guarantee is given about the signature of methods here (the perimeter method in Rectangle doesn’t adhere to the signature defined by the ABC), the interpreter only checks that abstract methods of the parent class have been implemented in the child class. Worse yet, it doesn’t guarantee that perimeter is even a Callable object.

Another way is to do it through structural subtyping, through protocols, but they come with additional headaches when it comes to debugging code, because the contract is enforced implicitly through developer-enforced adherence to an object shape rather than the language itself.

3. Python’s strange tooling story

This one is the most difficult to comprehend when programming in Python, the reality of a 30+ year-old language having mediocre tooling.

The issues with Python’s tooling most likely emerges from the fragmentation of its huge community and the dire need for backwards compatibility which, in practice, seems to actually be slowing down any form of progress. This feels like the Python 2 to Python 3 transition has somehow scarred core developers forever and made the whole community wary of any type of transformative change.

To accommodate every single niche need, Python ends up having several solutions for every problem

• type checking: mypy, pyright, pyre, pylance, pytype
• linting: pylint, flake8, yapf, ruff
• formatting: black, autopep8, pyfmt, isort, yapf
• package management: pip, poetry, conda, uv
• etc.

While this fragmentation (or competition) exists in other languages, Python is an outlier in that, until recently, it hadn’t really translated in clear gains in terms of usability, performance or quality.

The emergence of astral as a producer of high-quality, high-performance tools for the Python ecosystem (ruff and uv so far) has been somewhat of a game-changer in that regard in the past year or so, which leads me to believe that things might change for the better.

4. Python’s performance

Python’s dynamic typing, while offering flexibility and ease of use, is often cited as a factor contributing to its relatively slow performance. The interpreter’s need to infer variable types at runtime can introduce overhead, leading to slower execution compared to statically typed languages, especially for compute-intensive tasks where Python often has to fall back to relying on C extensions.

However, it’s essential to note that the performance gap between dynamically typed and statically typed languages isn’t solely due to dynamic typing but also because of Python’s interpreted nature and runtime overheads.

Indeed, and despite very similar challenges, advancements in JavaScript’s performance over the years have showcased that performance improvements are indeed possible in the realm of dynamically typed languages, given enough investment and competition between big actors.

Still….

There are other things that are not quite optimal in the whole Python experience:

• support for asynchronous programming still feels bolted on ten years later and isn’t as sturdy, battle-ready and feature-rich as the JavaScript version (Promise / async / await)
• default arguments are still an issue when using mutable types, leading to bugs and data corruption
• the language strangely still doesn’t have symbols, unlike JS, Ruby, Clojure, Elixir or Rust

And still… we have reasons to remain very hopeful for the future of Python.

Just as it was the case for Javascript in the 2000’s and the massive investments by Google, the Mozilla Foundation, Microsoft, Apple and others, the language is receiving a lot of attention and funding, because some of its inherent limitations are starting to become somewhat of a hindrance as the adoption rates keep going up.

As a result, work is being done to improve type management in the language (most recently in the 3.11 and 3.12 releases) and boost performance and concurrency through attempts to remove the GIL and maybe add a JIT compiler to the language.

Those efforts might not yield the expected results immediately, they might even go nowhere, but they are a welcome and exciting development, sending a clear signal that Python will not rest on its laurels and will keep on evolving as a language.

On the ecosystem side, libraries are being released along the way that address shortcomings of the language (see the influence of trio on the whole asynchronous ecosystem) and the quality of tooling is receiving a major lift from private initiatives (see astral.sh).

All of that work is the result of constant tectonic movements in the programming language community and the associated beneficial cross-pollination of ideas (JIT, GIL removal, gradual typing to name a few things that were topics of experimentation in other languages in the past few years).

The quality of programming languages is always a matter of trade-offs and compromise, especially languages whose 2.0 release happened more than twenty years ago.

Despite some of the issues mentioned in this article, Python remains as capable, adaptable and expressive as ever and the promises shown by recent evolutions in its wider ecosystem should keep us hopeful that it will stay relevant as the lingua franca of engineering in the near future.