How to turn ugly Java APIs into elegant, type-safe Scala APIs
One of the reasons people use Scala is to have access to vast amount of Java libraries.
While it’s very beneficial to have a library for almost everything, these libraries are written in Java, which means you let in all the limitations of Java you have been escaping from — greatly restricted type-safety, lots of verbosity etc. Fortunately, it’s not very hard to mask these deficiencies with Scala wrapping that will ease the pain of consuming such an API.
But sometimes these Java APIs utilize an object-is-just-like-a-hash-map technique which you can recognize by having lots of methods either operating on
Map<String, Object> or, in the so-called fluent style, having signatures like
withYyy(String name, Object value) etc .
This tendency destroys all the type-safety, maintainability, ability to refactor your code-base and can introduce a lot of subtle bugs, because you need to keep track in your head what instance is stored at which key, throughout all the usages of the library. Unfortunately, this is not something that can easily be gotten rid of in Scala — as you do not have any means of introspecting a type in pure Scala — apart from reflection.
On the other hand, powerful generic metaprogramming is achievable in Scala and there are libraries that provide all these functionalities, like the renowned Shapeless.
Although, it has the, somewhat unjust, reputation of being prohibitively hard to use, I think that for the purpose of getting rid of traces of Java-esque APIs, it is just fine. In the course of this post, I’ll be explaining how to do it systematically.
As an example of such approach, let’s take a java-jwt library from Auth0. This is a great library for JWT but when it comes to type-safety it has all the drawbacks you’d expect from Java library. Encoding and decoding a token looks like this:
Or, alternatively, you can get and set all the claims as, you guessed it,
Claim isn’t anything you can, for instance, pattern-match on; it’s just an opaque interface with bunch of
asMap methods. So, even though the library itself is terrific, in Scala code-base it stands out like a sore thumb. What can you do about it? Ideally, you’d like to be able to have a type, say
that encapsulates all the information you might want to put into a JWT token. Of course, you’d have to make sure that private claims of type
C can be safely (and automatically) encoded and decoded from a token. So, let’s code these assumptions first:
These signatures tell us that encoding/decoding a JWT token is possible if and only if we know that the type
C that we’re using as private claims is actually a valid
Claim. Proof of this fact is obtained by providing an implicit
This typeclass knows how to encode and decode its instances, given some internal representation named
Repr. In other words, it says that for some type
Of, given its JWT representation is
Repr, we can:
Reprinto JWT (via
- decode JWT token to
You might be wondering why
Repr is not a type parameter but a type member? It’s because it doesn’t have to be explicitly exposed to
Claims consumers. As far as they’re concerned,
Claims is a type constructor yielding the type of correct JWT claims given an
Of type. When you have
Claims[Foo], you know that
Foo can be converted into a JWT token and retrieved from it. What you do not need to know is how
Foo is represented in this process. This is relevant only to implementations of
Claims, thus it is “internal” to the
ClaimsDecoder will be used as a wrapper over the underlying Java library that will translate some types to appropriate
To implement these traits, you’ll need to have info about type member names and types. That’s what we can use Shapeless for. Let’s take a small detour on what Shapeless provides.
One of the features of the library that we will undoubtedly need is called labelled generic. In short (based on examples from Dave Gurnell’s The Type Astronaut’s Guide to Shapeless)
To comprehend all the details, you might want to read the excellent Dave’s book— but to get our API done we need to understand just two points:
a class can be generically represented as a
HListof its field types
a field type consists of the pair of type and literal type representing field name — that’s the
Symbol @@ “name”part. Please note how literal is encoded in the type — hence the name literal type. Such a type has only one member — the constant String, like “name” or “numCherries” (and no other)
In addition to convenient
HListrepresentation, labelled generic provides
frommethods which switch between an instance and its
Armed with this knowledge we can now go back to the API example.
So it’s turned out that we can build generically
Claims for any type that:
- has labelled representation in form of an
- elements of this
Now, we need to create an implementation of encoders/decoders for
FieldType[T, Symbol @@ literal]. Dealing with implicits based on recursive types is easy — you must always follow the type structure.
HList can be either:
- empty — represented by singleton
case object HNil, or
- concatenation of _head_ element of type
HList— represented by
case class ::[+H, +T <: HList](head : H, tail : T)
So you need to provide implicit instances for each case:
- in empty case you, obviously, don’t have to decode/encode anything
- in non-empty case you have to be able to encode/decode the head (which will be a
FieldType) and then, recursively, process the tail
The only missing part is the one dealing with fields itself. To encode/decode a field, we need to know its name and its type. Furthermore, the type of the field must be something that our underlying library supports. As we’ve seen — names are encoded as literal types. To get the runtime information we want from such a type, we need to delve into Shapeless once more
Literal types will become a first-class Scala citizen in 2.13 release, but before this happens libraries have to invent their own ways to expose them to the programmer. In Shapeless this is done with help of the type called
Witness. To quote Dave Gurnell’s book once again:
In other words, to get a runtime value out of literal type, we need to demand an implicit
This being solved, we have to take care of encoding and decoding supported types. Now, what constitutes a supported type? The documentation of
java-jwt states that:
Currently supported classes for custom JWT Claim creation and verification are: Boolean, Integer, Double, String, Date and Arrays of type String and Integer.
These types have nothing in common, so we need to use ad-hoc polymorphism to model JWT-enabled types. We’ll need a type-class for this:
If we were confined to types that the original library supports, it’d not be that good — for instance, Scala collections, or
Option are not supported. But this mechanism is really extensible — to support new types we just need to provide new instances of
… and, voila,
Seq will work. We can even add a generic mechanism for adding new types:
With this mechanism in place, we can formulate the final missing bit — encoding/decoding single field types ie. we can decode a field, if we have a
Witness of its name, and there exists a
PrivateClaimType for its type.
Now you can easily use the Java API in type-safe way. For instance:
If you happen to use Java library with not-so-type-safe API, you should try to convert it to type-safe Scala code. You can try to implement it as outlined in this post:
- create type-classes describing capabilities you need (eg.
- use Shapeless generic variants to convert Scala types to a representation you can operate on,
- provide instances of your type-classes which translate between type structures and Java library calls (with fine granularity),
- write implicit layer that works on generic representation and summons instances of type-classes based on introspection.
The full code for this article can be found on GitHub