Permutive Community Engineering, September 2019

This article gives a summary of projects that I worked on in my second month on the Community Engineering team at Permutive. For more context about these reports, please see last month’s edition.

Cats 2.0.0

One of the projects that I had focused on in August was preparation for the Cats 2.0.0 release, which finally happened this month. We ended up publishing one last release candidate with a couple of changes that broke source compatibility with previous releases (but that we thought were important enough to delay the release for a few days), and I made one minor change (and reviewed a couple others) after that release candidate.

I spent much of the rest of the month on follow-up tasks related to the 2.0.0 release, including three blog posts: one on how we managed to fix theParallel type class, one on why Cats users should care about Parallel to start with, and one more general Cats 2.0 migration guide. I also updated and published new versions of most of the open source libraries I maintain that depend on Cats, including Catbird, which provides type class instances for Twitter’s Util and Finagle libraries, iteratee.io, and Circe, together with most of the other libraries in the Circe organization (see the section below for details).

I also opened several Cats issues to track problems that I’d noticed during the preparation for 2.0.0 but wasn’t able to fix.

Cats and implicit scope

In last month’s report I mentioned an experiment that involves moving the standard library type class instances in Cats into what Scala calls “implicit scope”. I worked through several iterations of this project in September, starting with a pull request and including a blog post that goes into more detail about the proposal’s motivations and benefits. I’ve also published the experiment to Maven Central, so that other developers can try it out more easily in their own projects.

We’re still months (or more) from this change possibly becoming a reality in Cats, but so far the improvements to usability and compile times are promising, and we’re in a good position to be able to get feedback from a wider set of users.

Circe 0.12

Circe depends on Cats, and our latest release series (0.12.x) had been waiting on the final Cats 2.0.0 release for almost five months (we ended up with four milestones and four release candidates during that time). I published 0.12.0 the day after Cats 2.0.0 came out, and we almost immediately noticed that I’d introduced a pretty serious bug since the last release candidate. Fortunately the bug only affected one deprecated method, but we still fixed it in an 0.12.1 release on the same day.

We published new 0.12 releases for most of the other libraries in the Circe organisation, including circe-fs2, circe-optics, circe-jackson, and circe-derivation, with the circe-derivation release in particular featuring some improvements over the previous milestone. I also moved the circe-generic-extras module out of the core Circe repository so that it can be separately developed and versioned, and I created and published a fork of sangria-circe named circe-sangria, since it makes more sense for this integration module to live in the Circe organisation, and since we’re still working on getting access to the Sangria repositories.

I’ve also been working on several changes for the first Circe 1.0.0 milestone release, and wrote a blog post about one of these threads. I was hoping to publish this milestone in September, but I‘m revising that to “maybe October” 😃.

Scala 2.13 migration

Last month I wrote that “we’re not likely to be in a position to use 2.13 in production for at least a couple more weeks”, and we actually hit this target, although only for a canary deployment of the main Permutive API service. We’re currently still waiting on non-milestone releases of http4s and Sangria before moving over to 2.13 more widely, although we’re cross-building many of our internal libraries now that Cats 2.0.0 is out, and we’re fairly confident that the rest of the transition will be straightforward.

Other tasks and projects

• Updated all of my Sangria 2.13 pull requests, adding MiMa checking for binary compatibility so that we can get the versions right once we’re able to publish releases.
• Published new releases of Permutive’s own fs2-google-pubsub and sbt-liquibase-plugin.
• Wrote up a series of Circe questions and answers on Stack Overflow (1, 2, 3, 4).

Other notes

Last month I introduced some categories for estimating how I’d allocated my time, and the distribution this month looks similar:

• 20% purely internal work (code review, planning, coding challenge review, etc.)
• 20% open source work that’s directly motivated by internal needs
• 20% open source work that will have benefits for internal projects
• 40% open source work with little or no internal motivation

I’m putting the five blog posts I published this month into the last category, which is the main reason it’s a little higher than last month, but I think there’s an argument that they fit better in the third (this might be an indication that the framework needs some reworking).

October preview

• Circe 0.12.2 release with a few useful additions to 0.12.1
• More blog posts
• First Circe 1.0.0 milestone
• Scala 2.13 in production at Permutive (but more this time)
• New circe-json-schema library