specs2 is an open-source project which started more than 10 years ago now. What started as a small experiment with Scala turned to a fertile playground for learning functional programming, getting work opportunities, meeting great people around the world. Like many other open-source developers I spent countless hours working on it at nights, week-ends, in public transit, where ever I could find a power source and even while dreaming 😁. As a result specs2 has accumulated a number of features which are not worth their weight and which can be revisited now.
The main idea
One thing which sets specs2 apart from other test libraries is its data model. At its heart it is a stream of “fragments” where a
Fragment is a “description” and an “execution” which can return a “result”. By varying the different ways to create those elements we can provide many useful features for testing:
- “literate” tests where the emphasis is placed on some specification text annotated with test code
- hierarchical tests (test suites as a tree of tests, the more classical way of writing tests)
- a whole box of matchers, including data tables
- different ways to execute tests: sequentially, concurrently, with “join points”
- filtering, by name, with tags etc…
- many different reporters
- set-up and tear-down actions
- data-driven tests
- integration with property-based testing
- and much more…
When too much is too much
Unfortunately there are some features which I thought interesting at the time which are either not really used (or even known!) or which bring a massive complexity tax in the implementation. And talking about implementation, it can also be trimmed down since specs2 currently embeds a big part of the
eff library, which is a fairly complex one.
This helps no one. Neither the occasional developer who would like to implement a new feature nor myself when such a bug occurs: on the surface it looks like things have been working for a long time only thanks to a Scala bug! Unfortunately debugging that issue is non-trivial, the implementation is 99% complex for a feature which is used 1% of the time.
On the other hand, after all this time, one major feature is still missing in specs2: the ability to run several specifications which will open resources only once, like a database connection, and close them safely at the end of the run. I think I know how to do this but once more the implementation gets in my way.
So I think it is time to be reasonable and revisit some of specs2 features.
Trimming down the mastodon
Beware: breaking changes coming up! Removing esoteric features will possibly impact some users out there and I will provide a migration plan. I hope that the impact on a few users will be worth the gained stability for all the rest.
Here is a laundry list of things I want to remove or simplify.
In terms of features:
- isolable specs: that makes the implementation of mutable specifications so complex, it has to go. Collateral damage: if you use
AllExpectations(another less-known feature) your specification will need to run sequentially
specs2-formI’ve had this dream of implementing a Fitnesse on steroids using specs2. I still think there is a need for this kind of tool but short of being able to devote a massive amount of time to it and dog-fooding it for something real, that’s just dead weight on the project
specs2-gwtI have never been a fan of the
Given/When/Thenformat for writing specifications. It is inherently mutable and I have tried 3 different ways to support it, the 3rd being closer to a functional approach with much complexity. This should be declared a failed experiment now
specs2-mockwith the good work of Bruno Bonanno on mockito-scala this module is no longer necessary and I can be finally forgiven for the sins I’ve been guilty of for trying to adapt Mockito to Scala
specs2-analysisthis module does not hurt but I also don’t think there’s anyone on the planet using it!
- scopes and
DelayedInitwas a good trick to create reusable scopes and save a few characters. Unfortunately Scala 3 will probably not have them. I don’t think it is worth lobbying the Scala team to complexify the Scala compiler when there are reasonable alternatives
- auto-numbered / named examples. This feature was an attempt at mitigating one drawback of “immutable specifications”: having to come up with method names for each example. Eventually it turns out that it is worth having a proper method name since it helps with the navigation between the text and the code
Some of these features are really tied to the core of specs2 and will go (isolable specs, scopes), I am happy to donate the code for other features to be maintained in separate projects if anyone wants to (the various specs2 modules).
Now in terms of implementation:
effcan go. This is totally overkill. I can implement a much smaller datatype providing the necessary features: effectful code, async actions and bracketing for resources
- the internal components in specs2 use trait stacking and in the light of my recent experiments with dependency injection I think that regular classes + interfaces + delegation will work as well if not better
- the JSON matchers implementation is atrocious. I think that more disciplined optics should be used to be the base of those matchers. This is a lesser priority though since this is very localized complexity
- I would love to see if I can rework the trait stacking in the
Specificationtrait to limit it to the only places where it is absolutely necessary so that most of the features can be accessed through imports rather than with stacking traits
build.sbtfile can also get some love. It will be simplified if some modules go away but even more simplification would be nice
Another thing I might do at some stage is move some modules to their own projects so that they can evolve at their own pace and don’t get in the way of new specs2 releases when there is a new Scala version:
specs2-catsmatchers for cats data structures
specs2-scalamatchers for scalaz data structures
specs2-shapelesssupport for “diffing” case classes using shapeless
Feedback is welcome!
I encourage specs2 users and maintainers to give me their feedback on that plan, even small encouragements are welcome since it is a lot of work ahead!