Make Your Scala Compiler Work Harder

Introduction

The Scala compiler and type system already work hard to try and catch problems in code before they reach runtime. The standard compiler configuration is a balance between compile speed, utility and correctness. If you are willing to dig into the extra options offered by the Scala compiler, you might find you catch more real problems before they reach your runtime.

The configurations outlined in this article involve weighing the benefits and costs of slowing down the compiler but catching bugs and errors that it otherwise wouldn’t. At Hopper, we are currently testing enabling these options with some of our services to determine whether having these enabled is going to have a net benefit to customers in the app.

The information included in this article is relevant to the Scala 2.12 compiler. Options and flags for Scala 2.13 have changed, see here for information on those changes. The same ideas should work in 2.13 but the implementation details will differ.

Acknowledgements

My friend Rob Norris (@tpolecat) is to thank for my initiation experimenting with extra scalac options. This is a good guide to some of the flags available. It digs into some of those options and discusses how to make them work with your development environment, and how to deal with the occasional false positives that they can bring to light.

Getting Started

If you work in a team development environment, it’s likely that your build tool configuration (e.g. build.sbt) already has a few scalac (Scala compiler) options defined.

For example, if you dig into your build files, you might see flags like:

.settings(scalacOptions ++= Seq(
  "-Ywarn-unused:imports",
  "-Xfatal-warnings",
  "-deprecation",
  "-feature",
  "-unchecked"
)

These are common modifications to the standard compiler settings, but they only scratch the surface of what is available. -Xfatal-warnings in particular is one that causes the compiler to fail if there are any warnings. This is a popular setting for many projects, but it can cause problems when you are trying to add new warnings and fix them up in an incremental fashion.

Another problem is that using -Xfatal-warnings with -deprecation means that any deprecations will cause the compiler to fail. This does reduce the usefulness of using -deprecation since the code will no longer compile with deprecations, at which point the @deprecated buys you nothing over just getting rid of the method or class.

If you do want to keep -Xfatal-warnings and still have the deprecation cycle work, take a look here for information on how you can write a custom reported to enable this behavior. Further discussion on this limitation can also be found here.

In order for us to start experimenting with new compiler warnings, we need to be able to do a couple of things. Firstly, we need to introduce the new warnings, but not affect other developers while we fix them. Secondly, we need to turn off the fatal warnings (if turned on) so that we can fix the extra warnings at our own pace and keep things working while we do.

Both of these can be achieved by introducing a scalac options configuration specific to your environment (these could go in a .gitignore file, but I choose to enable mine system-wide under my user SBT settings):

Start by creating scalacOptions.sbt in either your project (gitignored) or under ~/.sbt/1.0/ with at least

scalacOptions -= Seq(“-Xfatal-warnings”)

in it, which will stop warnings being fatal as we go through and fix them (assuming that fatal warnings are turned on in your build, if not you can skip this setting).

In the same file, you can now introduce the new warnings you want to use. Here’s one of mine:

scalacOptions in Compile ++= Seq(
  "-deprecation", // Warning and location for usages of deprecated APIs.
  "-encoding", "utf-8", // Specify character encoding used by source files.
  "-explaintypes", // Explain type errors in more detail.
  "-feature",  // For features that should be imported explicitly.
  "-unchecked",  // Generated code depends on assumptions.
  "-Xcheckinit",  // Wrap field accessors to throw an exception on uninitialized access.
  "-Xlint:adapted-args",  // An argument list is modified to match the receiver.
  "-Xlint:by-name-right-associative",  // By-name parameter of right associative operator.
  "-Xlint:constant",  // Constant arithmetic expression results in an error.
  "-Xlint:delayedinit-select",  // Selecting member of DelayedInit.
  "-Xlint:doc-detached",  // A detached Scaladoc comment.
  "-Xlint:inaccessible",  // Inaccessible types in method signatures.
  "-Xlint:infer-any",  // A type argument is inferred to be `Any`.
  "-Xlint:missing-interpolator",  // A string literal appears to be missing an interpolator id.
  "-Xlint:nullary-override",  // Warn when non-nullary `def f()' overrides nullary `def f'.
  "-Xlint:nullary-unit",  // Warn when nullary methods return Unit.
  "-Xlint:option-implicit",  // Option.apply used implicit view.
  "-Xlint:package-object-classes",  // Class or object defined in package object.
  "-Xlint:poly-implicit-overload",  // Parameterized overloaded implicit methods are not visible as view bounds.
  "-Xlint:private-shadow",  // A private field (or class parameter) shadows a superclass field.
  "-Xlint:stars-align",  // Pattern sequence wildcard must align with sequence component.
  "-Xlint:type-parameter-shadow",  // A local type parameter shadows a type already in scope.
  "-Xlint:unsound-match",  // Pattern match may not be typesafe.
  "-Yno-adapted-args",   // Do not autotuple.
//  "-Ypartial-unification",  // Enable partial unification in type constructor inference
//  "-Ywarn-dead-code",  // Warn when dead code is identified.
  "-Ywarn-extra-implicit",  // More than one implicit parameter section is defined.
  "-Ywarn-inaccessible",  // Inaccessible types in method signatures.
  "-Ywarn-infer-any",  // A type argument is inferred to be `Any`.
  "-Ywarn-nullary-override", // non-nullary `def f()' overrides nullary `def f'.
  "-Ywarn-nullary-unit",  // nullary method returns Unit.
//  "-Ywarn-numeric-widen",  // Numerics are implicitly widened.
  "-Ywarn-unused:implicits",  // An implicit parameter is unused.
  "-Ywarn-unused:imports",   // An import selector is not referenced.
  "-Ywarn-unused:locals",   // A local definition is unused.
//  "-Ywarn-unused:params",  // A value parameter is unused.
  "-Ywarn-unused:patvars",   // A variable bound in a pattern is unused.
//  "-Ywarn-value-discard",   // Non-Unit expression results are unused.
  "-Ywarn-unused:privates"  // A private member is unused.
)
scalacOptions -= Seq(“-Xfatal-warnings”)

(Once again thanks to Rob Norris for curating most of these useful flags and saving me the effort).

You will notice some of these are commented out right now. This is part of the phased nature of fixing the warnings, and in at least a couple of cases, they will probably be left disabled. For example, while I appreciate that relying on numeric widening is not in the strictest sense correct, it’s also very common and I think is a little pedantic to call out and make people fix everywhere. (This is where an Int is passed to a method expecting a Double for instance, and most languages automatically widen types under those circumstances).

I also have not turned on any of the language extension feature flags (like -language:higherKinds or -language:postfixOps). This is a personal preference, but I like to see these warnings telling me about the use of those features. Higher-kinds warnings can still be silenced with an import in the file where the feature is used. Postfix operations are sometimes trouble, and I would rather just say e.g. 5.seconds than enable postfix ops to write 5 seconds but potentially get a confused compiler when semicolon inference fails.

Warnings in Detail

Below are some of the warnings we just enabled, with examples of what they might find.

This list is not exhaustive, some of the examples are hard to reproduce in small snippets of code, but hopefully, you will find the ones mentioned here entertaining and informative.

Note that I (mostly) use Metals and VSCode for Scala development, and the warnings we enabled integrate nicely with that experience once the metals sbt import is refreshed.

-Xlint:adapted-args, -Ywarn-adapted-args or -Yno-adapted-args

-Yno-adapted-args may be preferable after you have cleared out the warnings, as it prevents accidental auto-tupling which is often a bug (and you can make tuples explicitly using an extra set of parens if you want them)

E.g.

-Xlint:by-name-right-associative

A bug in the current scala compiler means that in-fix invocation of right-associative methods (those ending with :) taking by-name parameters will eagerly evaluate the by-name before the method is invoked, effectively passing them as an evaluated value. This lint warning catches that (rare) scenario:

-Xlint:constant

Warns you if the compiler detects that a constant definition would result in an error when it is evaluated:

(agreed, this is a dumb example, but it has caught real problems in the past, e.g. null pointer exceptions, etc.)

-Xlint:infer-any

One of my favorites, this catches when a method invocation results in a generic type being inferred as any. This is a fairly common bug, e.g.:

if you check this and the code is correct (it isn’t in this example), you can add the type parameter explicitly to stop the warning, e.g. xs.contains[Any](“hello”) — this also communicates to a reader that this is intentional.

-Xlint:missing-interpolator

Come on, admit it; you’ve done this before 😃

-Xlint:private-shadow / -Xlint:type-parameter-shadow

Similar concepts to uncover suspected accidental shadowing/hiding of a variable or type from another scope, another one that catches a lot of mistakes (particularly the type parameter shadowing), e.g. (in this case, the method probably wants to refer to the class T instead of introducing its own which will be a whole new type):

-Xlint:delayedinit-select

A rare but nasty initialization bug can result from referring to a value in a delayedinit section of an App object. E.g.

-Xwarn-unused:[implicits, imports, locals, params, patvars, privates]

A set of detectors for unused variables in their various forms that can be hiding real bugs, e.g.

Not a serious bug, but the intention here was likely to use err and not error in the logged message (or println in this case). The warning tells us that the Error message was extracted but not used, potentially indicating a problem. If you want to get rid of the message, use case Error(_) => instead (or in this case, using err would likely be more correct).

Unused imports detects any imports that can be removed. It seems to work better than some IDE detectors which can yield false positives, particularly for implicit imports:

What’s Next?

As already mentioned, some of the warnings in my scalac options file are commented out right now. After you enable the extra warnings flags, you will likely find a lot of warnings coming up and it’s now time to fix them, and you might find some of them are false positives and need extra effort to silence the warnings (which you will need to do if you want to turn fatal warnings back on at some point).

In the next article, we’ll take a look at some of the warnings that are not yet enabled, and how you can enable them and still get to zero warnings with a bit more effort for those few cases where they warn on valid code. If you are determined, there’s usually a way to quiet the warnings but still keep them turned on to catch real bugs.

Turning on the extra warnings in your build environment is not going to catch every bug you write, nor even a majority of them, but it could well catch some (and in my experience, has) before they get any further. It’s a reasonably low cost for some potentially big savings, and you can always choose to turn the warnings back off again once you have fixed the ones you feel you can do something about, though the real value comes if you can leave them on for all future builds.

Stay tuned for another article in this series with the results from our experiment with these options enabled in our Scala compiler.

PS — we’re looking for talented Engineers to join our team. Hopper is hiring! Check out our postings here.