How to turn ugly Java APIs into elegant, type-safe Scala APIs

One of the reasons people use Scala is to have access to vast amount of Java libraries.

While it’s very beneficial to have a library for almost everything, these libraries are written in Java, which means you let in all the limitations of Java you have been escaping from — greatly restricted type-safety, lots of verbosity etc. Fortunately, it’s not very hard to mask these deficiencies with Scala wrapping that will ease the pain of consuming such an API.

But sometimes these Java APIs utilize an object-is-just-like-a-hash-map technique which you can recognize by having lots of methods either operating on Map<String, Object> or, in the so-called fluent style, having signatures like withXxx(Object value), withYyy(String name, Object value) etc .

This tendency destroys all the type-safety, maintainability, ability to refactor your code-base and can introduce a lot of subtle bugs, because you need to keep track in your head what instance is stored at which key, throughout all the usages of the library. Unfortunately, this is not something that can easily be gotten rid of in Scala — as you do not have any means of introspecting a type in pure Scala — apart from reflection.

On the other hand, powerful generic metaprogramming is achievable in Scala and there are libraries that provide all these functionalities, like the renowned Shapeless.

Although, it has the, somewhat unjust, reputation of being prohibitively hard to use, I think that for the purpose of getting rid of traces of Java-esque APIs, it is just fine. In the course of this post, I’ll be explaining how to do it systematically.

As an example of such approach, let’s take a java-jwt library from Auth0. This is a great library for JWT but when it comes to type-safety it has all the drawbacks you’d expect from Java library. Encoding and decoding a token looks like this:

Or, alternatively, you can get and set all the claims as, you guessed it, Map<String, Claim>

Of course Claim isn’t anything you can, for instance, pattern-match on; it’s just an opaque interface with bunch of asBoolean, asMap methods. So, even though the library itself is terrific, in Scala code-base it stands out like a sore thumb. What can you do about it? Ideally, you’d like to be able to have a type, say

that encapsulates all the information you might want to put into a JWT token. Of course, you’d have to make sure that private claims of type C can be safely (and automatically) encoded and decoded from a token. So, let’s code these assumptions first:

These signatures tell us that encoding/decoding a JWT token is possible if and only if we know that the type C that we’re using as private claims is actually a valid Claim. Proof of this fact is obtained by providing an implicit Claims instance.

This typeclass knows how to encode and decode its instances, given some internal representation named Repr. In other words, it says that for some type Of, given its JWT representation is Repr, we can:

• convert Of instance to Repr (privateClaims method),
• encode Repr into JWT (via ClaimsEncoder[Repr]),
• decode JWT token to Repr (via ClaimsDecoder[Repr]),
• convert Repr back to Of (of method).

You might be wondering whyRepr is not a type parameter but a type member? It’s because it doesn’t have to be explicitly exposed to Claims consumers. As far as they’re concerned, Claims is a type constructor yielding the type of correct JWT claims given an Of type. When you have Claims[Foo], you know that Foo can be converted into a JWT token and retrieved from it. What you do not need to know is how Foo is represented in this process. This is relevant only to implementations of Claims, thus it is “internal” to the Claims type.

ClaimsEncoder and ClaimsDecoder will be used as a wrapper over the underlying Java library that will translate some types to appropriate java-jwt calls.

To implement these traits, you’ll need to have info about type member names and types. That’s what we can use Shapeless for. Let’s take a small detour on what Shapeless provides.

Shapeless

One of the features of the library that we will undoubtedly need is called labelled generic. In short (based on examples from Dave Gurnell’s The Type Astronaut’s Guide to Shapeless)

To comprehend all the details, you might want to read the excellent Dave’s book— but to get our API done we need to understand just two points:

a class can be generically represented as a HList of its field types

a field type consists of the pair of type and literal type representing field name — that’s the Symbol @@ “name” part. Please note how literal is encoded in the type — hence the name literal type. Such a type has only one member — the constant String, like “name” or “numCherries” (and no other)

In addition to convenient HList representation, labelled generic provides to and from methods which switch between an instance and its HList representation.

Armed with this knowledge we can now go back to the API example.

So it’s turned out that we can build generically Claims for any type that:

• has labelled representation in form of an HList, and
• elements of this HList (FieldType) have ClaimsEncoder and ClaimsDecoder

Now, we need to create an implementation of encoders/decoders for HLists of FieldType[T, Symbol @@ literal]. Dealing with implicits based on recursive types is easy — you must always follow the type structure. HList can be either:

• empty — represented by singleton case object HNil, or
• concatenation of _head_ element of type H with an HList — represented by case class ::[+H, +T <: HList](head : H, tail : T)

So you need to provide implicit instances for each case:

• in empty case you, obviously, don’t have to decode/encode anything

• in non-empty case you have to be able to encode/decode the head (which will be a FieldType) and then, recursively, process the tail

The only missing part is the one dealing with fields itself. To encode/decode a field, we need to know its name and its type. Furthermore, the type of the field must be something that our underlying library supports. As we’ve seen — names are encoded as literal types. To get the runtime information we want from such a type, we need to delve into Shapeless once more

Shapeless

Literal types will become a first-class Scala citizen in 2.13 release, but before this happens libraries have to invent their own ways to expose them to the programmer. In Shapeless this is done with help of the type called Witness. To quote Dave Gurnell’s book once again:

In other words, to get a runtime value out of literal type, we need to demand an implicit Witness of it.

This being solved, we have to take care of encoding and decoding supported types. Now, what constitutes a supported type? The documentation of java-jwt states that:

Currently supported classes for custom JWT Claim creation and verification are: Boolean, Integer, Double, String, Date and Arrays of type String and Integer.

These types have nothing in common, so we need to use ad-hoc polymorphism to model JWT-enabled types. We’ll need a type-class for this:

If we were confined to types that the original library supports, it’d not be that good — for instance, Scala collections, or Option are not supported. But this mechanism is really extensible — to support new types we just need to provide new instances of PrivateClaimType:

… and, voila, Option and Seq will work. We can even add a generic mechanism for adding new types:

With this mechanism in place, we can formulate the final missing bit — encoding/decoding single field types ie. we can decode a field, if we have a Witness of its name, and there exists a PrivateClaimType for its type.

Now you can easily use the Java API in type-safe way. For instance:

Summary

If you happen to use Java library with not-so-type-safe API, you should try to convert it to type-safe Scala code. You can try to implement it as outlined in this post:

• create type-classes describing capabilities you need (eg. Claims, ClaimsEncoder, ClaimsDecoder),
• use Shapeless generic variants to convert Scala types to a representation you can operate on,
• provide instances of your type-classes which translate between type structures and Java library calls (with fine granularity),
• write implicit layer that works on generic representation and summons instances of type-classes based on introspection.

The full code for this article can be found on GitHub