Python Needs A Fresh Testing Framework
* pause for eye rolls *
tl;dr: I’m writing a new testing framework — Nimoy.
Python has a super rich and active ecosystem, and has been around for over 25 years so what could it be missing?
I’ll try to explain.
A few years back I was pretty involved in the Groovy community. Groovy is by far my favorite programming language, and my favorite product of Groovy’s wonderful community is the Spock testing and specification framework.
The people behind Spock have done an incredible job. Spock makes writing tests fun again.
For the past year I’ve been paying the bills by writing Python at the awesome Healthy.io, but I miss the Spock framework. pytest is great but it’s not the same and here are a few reasons why.
Spock Is For Specs
Spock is by definition a testing and specification framework. It nudges you into writing your features (test methods) as structured specifications. This is possible thanks to a convention of method blocks such as setup, when, then, and expect.
This convention helps you write features that maintain focus on the tested component. The feature blocks maintain a distinction between preparation, execution and validation.
Natural Language Is Everything
I’m a firm supporter of Domain Specific Languages. DSLs allow you express yourself in a language that’s as near as possible to the one that’s natural to you. Spock’s DSL is very expressive in this way. You’ll notice the lack of boilerplate code, making tests short and readable.
That single expression under the then block is Spock’s way of stating expectations. In this case “Expect a single invocation of the request method with ‘payload’ as a value”. Succinct!
First-Class Data Driven Feature Support
Many times have I had to run the exact same test method multiple times with different values. In most frameworks there’s no organic solution for this issue. I usually end up duplicating the method or running inner loops for different values. Spock’s DSL for writing data driven features is elegant and beautiful.
Now we can think of the feature as a template. The where block allows us to declare variables and a set of values for every variable. The same feature method will run for every set — reducing clutter and complexity.
… And this is only the tip of the iceberg that is Spock.
So What Does This Mean For Python?
Python needs a Spock and I’m going to try and make one.
We’ve named the project “Nimoy” (still source, no official build) and I’m going to make it as close to Spock as I can. Hopefully, more blog posts describing the technical challenges are to come.
I’ve based Nimoy on the unittest module so that it will need no extra dependencies. This will also make use of the great tooling support that unittest already has.
In the meantime, I’ve managed to implement:
- A basic auto-discovering test runner
- A DSL for specification structure
- Lean assertion expressions
- Data-driven features
This is a valid, running specification:
I’m super excited about this project. I hope it has the potential to improve the lives of many Python developers.
