Permutive Community Engineering, October 2019

Travis Brown
Nov 4 · 3 min read

The community engineering team was at 75% capacity in October (I was out for a week of vacation), but we were able to wrap up several projects and kick off a few new things.

Sangria

A couple of weeks ago we were finally able to recover publishing rights for the Sangria project (see the August edition of this report for context). I had met earlier that week with developers from Twitter, the New York Times, and Commercetools to discuss the future of the project. Once we had recovered publishing rights, Yann Simon and I created a temporary new GitHub organization, forked the 18 repositories containing Sangria-related libraries, and started updating and publishing them with Scala 2.13 support. By the end of the day we’d published new releases for about half of Sangria’s modules, as well as a 2.0.0-M1 milestone release of Sangria itself.

Since then we’ve published releases for all but one of the remaining Sangria modules, and have started planning for the final 2.0.0 release, which we’re hoping to have ready in the second half of November. At Permutive we’re in the process of updating our internal GraphQL code to use the new Sangria 2.0.0 milestone, and we’re reaching out to other adopters to encourage them to do the same.

JSON Schema validation for Circe

We launched three new Circe libraries in October, including circe-json-schema, which I mentioned in last month’s team report. We use the Everit library internally at Permutive together with Circe for JSON Schema validation, and circe-json-schema is a small project that simplifies interoperability between the two libraries and provides a minimal validation API that is consistent with other Circe modules.

Circe and recursion schemes

Our 0.12.0 release in September introduced circe-rs, an experimental module that provides a pattern functor for Circe’s JSON representation, which is designed for use with Scala recursion scheme libraries like Matryoshka or Droste. I’ve split this module out into its own repository on GitHub, and have added a new circe-droste module, which additionally provides the mapping Droste needs between the pattern functor and Circe’s JSON abstract syntax tree. This new circe-droste module isn’t published yet, because I decided to wait for a new Droste release that will include Scala 2.13 support, but we’re working on that.

Golden testing for Circe

The most significant of this month’s three new Circe libraries is circe-golden, which provides tools for writing “golden tests” for Circe’s encoders and decoders. This style of testing helps to address some common complaints about type class instance derivation (see the project description for details), and is something we’re planning to use extensively at Permutive to reduce boilerplate and give us more confidence in evolving our data models.

Bug fixes and new features for Circe 0.12

I also published a couple of new Circe patch releases (0.12.2 and 0.12.3) that include three bug fixes, some optimizations, and several useful API additions. While the bugs that have been fixed are minor and shouldn’t affect most users, we’re encouraging all Circe adopters to update to 0.12.3.

Scala 2.13 at Permutive

Most of our internal libraries are now cross-published for Scala 2.12 and 2.13, and many of our internal services are now running on Scala 2.13. Our new Sangria release unblocks one of the services that’s still on 2.12, and we’re hoping to finish that migration in early November.

We do have one service that is still running on Scala 2.11 because it uses the Spray framework, which is no longer maintained, and which has never been published for Scala 2.12. I spent some time this month migrating that service to Akka HTTP (Spray’s replacement), which will allow us to move directly to Scala 2.13.

Other tasks and projects

November preview

  • Several big releases (Sangria 2.0.0, Cats 2.1.0, Jawn 1.0.0)
  • Ongoing work toward Circe 1.0.0
  • Wrapping up Permutive’s Scala 2.13 migration
  • An overhaul for these team reports

Permutive

We build a real-time data platform that our customers use to power online experiences. We use Medium to write about the technology, product and community we work in.

Travis Brown

Written by

Permutive

Permutive

We build a real-time data platform that our customers use to power online experiences. We use Medium to write about the technology, product and community we work in.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade