plinth testing

Lets run code and see where it gets us!

This is a new installment in a blog series where we explore living and breathing Open Source projects.

Driven by curiosity, we searched a GitHub repository containing a Haskell package, for issues we could solve — to learn, build and who knows why else! After looking through contributors checklist we moved on to thinking about what exactly is the new functionality that needs to exist.

What we got is an LLM hallucination:

you can write a test that checks if all exported types have an `Arbitrary` instance.

This is at least partially True:

property: has Arbitrary instance

AI says:

Haskell: Ironically, you can use Haskell itself to parse Haskell modules. Haskell has a number of libraries for parsing, including parsec and attoparsec. These libraries provide a high-level interface for defining parsers.

A simpler approach would be to manually write test cases for each type exported by the module. This approach is more verbose but ensures that all types are tested correctly.

To test if tis true we need to come up with a list of Types the package exports. I was looking for a perfect way but I have to be mediocrates here: write a parser to process Haskell modules and return stuff in brackets, then list the modules by hand in the test.

What’s next?

I need to test the code and open a PR but the assertion will fail of course!

We keep diggin!