Are you sure your Scala 3 opaque types don’t instantiate?

Photo by Kelli McClintock on Unsplash

A while (cough 5 years cough) ago I wrote about a little-known issue with value classes/AnyVals in Scala 2.12. Here’s the article in question, which I obviously recommend to read. The tl;dr is that, in arguably the major practical usage cases, value classes will actually instantiate.

Now, after attending Magda Stożek’s presentation on opaque types at this year’s Scalar, I was inspired to write a follow-up.

The underlying problem with value classes stems from the circumstances of their origin. They were, inherently, an add-on to an otherwise established language dialect, with limited maneuvering space for modelling them on the JVM. Fortunately, Scala 3 eventually came along, and with it Opaque Type Aliases; a replacement for the value classes, able to benefit from the lessons learned from their conceptual predecessors.

In the interest of not being too clickbaity — opaque types do indeed mostly deliver on their promise of being an efficient abstractions. The “mostly” is caused by one exception, chiefly due to the limitations of the JVM. Let’s briefly explore how opaque types go about that.

Handling generics

Recall that, in the Finally — the “WHY” section of the original article, we started with the issue of instantiating the value class while decoding data from JSON, and drilled deep down enough to identify the ultimate problem: working with generics. This was the test code:

and the domain object itself:

Let’s go ahead and write the Scala 3 (specifically, 3.2.2) version of both:

Firing up jvisualvm shows us this:

No instances!

So far, so good, but we still have one question that needs answering: if the limitation of value classes flared up due to how the JVM handles references, what actually happens with opaque types?

For that, let’s look at the relevant parts of the second code snippet’s bytecode:

The fun stuff happens at the end of L0 in sanityCheckForDef. getThing gets invoked, and, normal for a generic parameter, on a type of Object, but which gets cast to a String, not any other "container" class. Before that, the Text "constructor" runs, but also returns a String. It’s clear that – to the runtime – the opaque type is completely transparent.

Observant readers might now point out the following: if the type of getThing's param is Object, then opaque types with an underlying primitive type might still have an instantiation problem. Let’s check that out then:

Take a look at the additional instructions in L0, particularly the boxToInteger.

So, yes, this is the one case with an additional overhead — the runtime needs to have the types reconciled, and the primitive value must be boxed. Still, there is no additional instance of a “container” type actually being instantiated, in contrast with value classes.

Additionally, note that the value is immediately unboxed, so that the actual type of text is int. The runtime semantics of opaque types are thus preserved.

But let’s say we’re not satisfied with that result. We do have one more trick up our sleeve — specialized :

That’s…​ odd. “That” being three things:

• our opaque type is still boxed;
• text2, which was obtained by invoking getThing on an Int directly, and added as a sanity check, is also boxed;
• finally, and most importantly, getThing lacking specialized variants in the bytecode – there’s only the original, Object one.

A brief search turned out this issue on the Scala 3/Dotty GitHub. The description and discussion implies that @specialized is simply nonfunctional in Scala 3. If that’s true, it’s a shame the documentation contains no indication of the deficiency.

OK, our final-final trick: @specialized is pretty much just syntactic sugar for polymorphic method definitions. There’s no reason why we can’t simply write those by hand:

Now, text2 uses the "primitive" getThing correctly, but not our opaque type text. I’m not 100% sure why this is the case, but I suspect it has something to do with the full definition of an opaque type alias, as quoted from here:

opaque type T >: L <: U = R
[…​] where the lower bound L and the upper bound U may be missing, in which case they are assumed to be scala.Nothing and scala.Any, respectively.

(emph. mine)

A circumstantial confirmation of being on the right track is that changing the definition to:

(and only Int, not AnyVal) makes the boxing go away. However, it raises two new problems:

• we’re making the opaque type “transparent” again, due to advertising the subtype relation externally,
• the compiler “loses sight” of the extension methods:

making an explicit type ascription to Numeric, like so:

removes the compilation error, but brings back the boxing.

This “sight” loss is probably due to how the “primitive” version of getThing forces the return type to Int (no idea why the type ascription works still). So, adding some subtype acknowledgement:

brings back the non-boxing bytecode. Unfortunately that still exposes the Int nature of our opaque type. And no, switching to opaque type Numeric <: AnyVal = Int doesn’t solve the problem – boxing comes back.

All in all, something very unexpected (for the record, I’ve also tried compiling on 3.3.0-RC3). Perhaps someone with a more inside look of the compiler would be able to explain what is going on, especially since this seems to be either an interplay of corner cases, or an oversight.

I’d go as far as to say this behavior violates the principle of least surprise — in the “specialization by hand” example, I’d expect getThing(text: Int): Int to be called, not the generic variant. The need to preserve type bounds is understandable, but if the compiler unboxes the Int after the call, and prioritizes Numeric 's semantics that way (and not the opaque type’s upper type bound), why not go all in? Imagine if there is a bug in one of the getThing methods – now we have it triggering for some Ints, but not for others.

But no matter how much we can complain and express perplexity, we still eventually need to write running code. And despite all the weirdness described above, the fact remains: there is always the option for the “subtyping workaround”, if heap usage optimization must be prioritized.

Summary

Here, at the end of our short investigation, we found out that:

• opaque types indeed do not instantiate in the vast majority of cases, working like most devs expected value classes should have had;
• opaque types do “instantiate” — or, rather, are boxed — in a certain corner case: when representing primitive types, and combined with generics;
• this “instantiation”/boxing constitutes identical behavior to actual, “bare” primitive types under generic parameter usage, and can theoretically be remedied in similar ways;
• practically, the workarounds are a bit more involved than expected, but still deliver — unlike with value classes.

In closing — and despite the relatively minor peculiarities discovered at the end — I hope that this blog post will convince those coding in Scala 3 to take a closer look at opaque types.

They are an excellent tool for improved type-based security of domain types, as well as a good option for creating lightweight object proxies (especially masking APIs from other JVM languages). And, for those not using Scala 3 yet, a point for making the transition.