Debugging implicits

Piotr Kukielka
Mar 23, 2015 · 5 min read

I’ve been recently asked what are my favorite and least favorite Scala features. Probably few options would be matching here, but I decided to give one answer for both questions: implicits. Why? Implicits are very simple and yet very powerful concept but with great power comes great responsibility. They can be used for implicit parameters, implicit conversions, view bonds and context bounds (type classes) letting us solve couple of different problems with one language feature. But at the same time we must be aware in what context we are using them and — which is much harder — which implicits are available in the current context.

Luckily we are not left alone with that problem. Main Scala IDEs like ScalaIDE or IntelliJ are able to list currently used implicits and we can do the same in REPL as well.blog

Let’s go through basic example with Ordering type class to check how things work together and what to do in case you are stuck. First, let’s create simple class wrapper around Int value and make array of its elements:

Now we want to sort our list using one of the methods from the standard library. We can do this using sortBy, sortWith, or sorted. We will use sorted:

Ouch, looks like we forgot about implicit value of type Ordering[IntWrapper]. Checking method signature would tell us the same thing:

We can fix that by bringing implicit value of type Ordering[IntWrapper] in scope, or by extending IntWrapper with class Ordered[IntWrapper] or Ordering[IntWrapper]:

That fixed our problem:

There is nothing difficult here. Required and provided implicit values have exactly the same type. Now let’s try how it works with inheritance version. To make things a bit more difficult we choose to extend our class with Ordered[IntWrapper] rather than Ordering[IntWrapper]:

And everything works again. But it raises one interesting question: how? Our method sorted requires one implicit parameter of type Ordering[IntWrapper]. We provided none, and we doesn’t even mentioned class Orderinganywhere. It would be reasonable to guess that some implicit conversions or implicits chaining happens behind the scene. Possibly one word of explanation what I understand by the term ‘implicits chaining’ is required. Martin Odersky in his book “Programming in Scala” says:

“One-at-a-time Rule: Only one implicit is tried. The compiler will never rewrite x + y to convert1(convert2(x)) + y. Doing so would cause compile times to increase dramatically on erroneous code, and it would increase the difference between what the programmer writes and what the program actually does. For sanity’s sake, the compiler does not insert further implicit conversions when it is already in the middle of trying another implicit. However, it’s possible to circumvent this restriction by having implicits take implicit parameters, which will be described later in this chapter.

Bold emphasis is mine. We will see example of behavior described by Odersky in a moment but first we need to introduce some methods to check what really happens under the hood. Except IDEs support mentioned at the beggining of this article we have at least few other options.

Compiler flag -Xprint:typer

To use this method you need to start Scala REPL with -Xprint:typer parametr. It will produce a lot of bloat which you can ignore. If we will use sorted method on the list we will see:

It actualy contains information about implicits, but it’s hidden in the background noise. Can you find it?

Additional downside of this method is that it will produce that kind of noisy output for every expression you enter.

REPL command :javap

:javap command gives us possibility to print various details of the bytecode generated by Scala compiler. Let’s try to use it to check what happens inside sorted:

Oh no, it’s even bigger then previous output. But I would argue that it’s easier to read. Output is a bit more structured and since we know we are looking for sortedList we can just search for it. Section which starts with public IntWrapper[] sortedList() contains what we need and have around 20 lines of code. Not so bad, but still far from perfect.

If you have problem with finding exact information about implicits, here it is:

We will analyze that in a moment, but first let’s look at our third — and most effective — debugging method.

Scala reflection API

We are almost there. As you can read in the scala reflection overview: “In Scala 2.10, a new reflection library was introduced not only to address the shortcomings of Java’s runtime reflection on Scala-specific and generic types, but to also add a more powerful toolkit of general reflective capabilities to Scala. Along with full-featured runtime reflection for Scala types and generics, Scala 2.10 also ships with compile-time reflection capabilities, in the form of macros, as well as the ability to reify Scala expressions into abstract syntax trees.”

Let’s try if we can use that new features to help us with debugging implicits:

Woha! That is exactly what we need without any bloat or syntactic noise. We can clearly see that what happened and in what order.

First Predef.refArrayOps implicit conversion is applied to convert Array to ArrayOps (which contains method sorted). Since sorted requires Ordering[IntWrapper] as implicit parameter Ordering.ordered implicit conversion is applied:

As you can see it requires another implicit argument (that’s what I called ‘implicits chaining’ before) which in our case should have type IntWrapper => Comparable[IntWrapper]. We haven’t defined it ourselves, but it already exists in Predef in form of $conforms[A] function:

IntWrapper inherits from Ordered[IntWrapper] (which in turn extends Any with java.lang.Comparable[IntWrapper]) and Function1 is covariant in it’s return type so IntWrapper => IntWrapper is actually subtype of IntWrapper => Comparable[IntWrapper] and types match. In fact we are passing to Ordering.ordered identity function.

Last thing worth to notice here is implementation of Ordering.ordered. It always creates new object which is nice to be aware of:

Conclusion

As we saw, mix of implicit conversions and implicits chaining could become hard to figure out without reaching help from IDE or compiler. Does it mean that implicits are inherently bad idea?

Not necessarily. Do we normally care what happens under the hood when we are using Ordering or other type classes? Or any other implicit conversions or views for that matter? Usually not. Implicits allow us to significantly reduce boilerplate code which would normally only obscure our view.

I guess it’s matter of taste. Some people prefer to have all that boilerplate generated, and then possibly hidden by IDE. Others prefer to have it generated by compiler which is IDE-agnostic, but that at first glance could be a bit harder to grasp and could look too ‘magically’.

Both approaches guarantees full control over the code, because we can always pass implicits explicitly or apply conversions by hand. We just don’t do that, because usually it’s so much easier to let the compiler do the dirty work.

Yes, in cases, you are lost and really need to debug implicits, you need to know your tools.

But is that really a bad thing?


If you are interested in Scala internals why don’t you take a look at our current openings and company atmosphere. We are always on a lookout for brilliant Scala engineers.

VirtusLab

Virtus Lab company blog

Piotr Kukielka

Written by

If you want to know a man, read his code.

VirtusLab

VirtusLab

Virtus Lab company blog