Zinc (sbt) friendly code

How to make you code compile faster with Zinc

In April I gave a presentation (50 Shades of Scala Compiler) at the ScalaUA conference. In the abstract, I promised to give some hints on how to write compiler-friendly code. The talk lasted only 40 minutes and Scala compilers have many, many shades (I almost forgot about Dotty!), so all the hints didn’t fit into the slides.

I don’t like false promises, so you can find first batch of hints (related to incremental compilers) below.

TL;DR;

  1. Don’t put more than one class/trait (and its companion object) in one source file.
  2. Make your packages small (5–7 sources at most).
  3. Add an explicit return type for all your public (or even all non-private) methods so they are not changed when is no such need.
  4. Make sure that your code is and will be free of unused imports.
  5. Don’t use wildcard imports from packages that often change. Use wildcards only if you must.
  6. Don’t mix implicits and regular public methods in one class.
  7. Macros are zinc-killers. Keep them separated and try not to recompile them too often.

Don’t use names if you don’t need them

If Bar.scala depends on foo from Foo.scala, then changing bar in Foo.scala will not make Bar.scalabe recompiled (even if Bar uses Foo).

How does Zinc know this?

This makes Zinc much faster in terms of files recompiled for a given change (less is recompiled, therefore compilation is faster), but also introduces new problems.

Generally, the following sections are focused on reducing the number of names used in order to speed up incremental compilations.

Less is more

The solution is as simple as possible: split your sources! If incremental compilation is not enough to convince you, you should be aware that it should also help with compilation time or even result in less conflicts during merges.

Types, we need more types!

If you don’t provide an explicit type, Scalac will infer one for you that is as precise as possible. Developers love this, but later on complain that compilations (even incremental ones) are long. What is the problem with methods without explicit types? The precise type generated by Scalac changes (sometimes often), even when we don’t want it to. Need an example? Here it is:

You may say that everything is fine as long as your code compiles. I agree if you don’t care how long compilation takes. Every time a type is changed, all usages need to be recompiled (and even more with an incremental compiler heuristic). Moreover providing explicit types in public members is generally considered as a good practice for making code more readable (especially without IDE support, e.g. during code review) and easy to maintain (especially in terms of binary compatibility).

In short, add return types to all public (or even all non-private) methods.
Do you want to improve compilation time? Adding return types to public members decreased compilation time in the Intellij Scala plugin by 17% (pull request).

Imports, wildcards and other nastiness

Let me show that if you care about your build performance (leaving maintenance concerns for another blog post), correct handling of imports is crucial.

Unused imports

Luckily, removing unused imports is quite easy. The simplest solution is to add an -Ywarn-unused-import flag to the compiler and -Yfatal-warnings on pull request validation builds. You will then be notified for every build that has unused imports. These warnings become an error on a CI build, which forces you to clean them up before the pull request is merged.

Why not set -Yfatal-warnings for all builds? Being forced to comment out imports, when you just want to check if e.g. using Vector will make your code faster, is really annoying in the longer term.

Of course there are tools that can manage imports for you: scalafix or your IDEs (IntelliJ, ScalaIDE and ensime)

Wildcards

Why are wildcard imports so hard for incremental compilers?
The first step to understand this evil is a simple test. Replace any wildcard import in your code with all members of that package. Later compile the code with ‘-Ywarn-unused-import’. I wish there were a compiler flag that would tell you how many imports from given wildcard are used :)

This is only the tip of the iceberg. With wildcard imports, an incremental compiler needs to become an ahead-of-time one. Why? Because you not only import all names that exist at the moment of compilation, but you also import all that will exist in future in that package. For an incremental compiler, the old python joke import jetboard from the.future has a new, bitter taste.

What can we do then? There is no silver bullet here but I can give you some hints:

  1. Make you packages (or generally import scopes) small. If your package has 5 sources instead of 20, it will make your code much, much easier to learn.
  2. Make more use of the private or private[your_package] keywords. I really wish everything in Scala were private by default (maybe someone should create a tool for that?).
  3. Use wildcard imports only from libraries (or pieces of code that are changed rarely). Wildcards are dangerous only when imported things change.

Implicits

This is why Zinc has to use a simpler approach for handling implicits. In short, if you use anything from class Foo and any implicit name from it changes, then our source is recompiled.
What does this mean? If you want smooth and incremental compilation, don’t mix implicits and normal code in a single class. Unless you are fine with tons of sources recompiled for no reason!

Macros

How can we live with that?
Either take incremental compilation of macros in your own hands and turn the recompileOnMacroDef flag off (in sbt/zinc), or try to remove all cases in which macros are recompiled. How to do so? First of all, place macros in dedicated files (containing only macro definitions) and clean up imports. Generally reduce things imported and used in sources containing macros.

Kudos and further reading

If you find incremental compilation interesting you can learn more from links below.

VirtusLab

Virtus Lab company blog

VirtusLab

Virtus Lab company blog

Krzysztof Romanowski

Written by

VirtusLab

Virtus Lab company blog