The Scala compiler is notorious for its long compilation times. This is considered one of the biggest problems in the Scala community at the moment. One of the main reasons for long compilation is that the Java HotSpot VM is not tuned for Scala programs and the Scala compiler is entirely written in Scala.
GraalVM performs significantly better than the standard Java HotSpot VM for Scala programs. In this blog post, we show how GraalVM can speed up compilation of Scala programs and thus improve productivity. All experiments in this article were ran on a two socket with Intel Xeon CPU E5–2699 2.30GHz machine with Linux installed. If you want to try it yourself, GraalVM is currently available for download on Linux and macOS.
For evaluating GraalVM for Scala compilation, we use the Scala Compiler Benchmark project. From this project, we compiled the source code of
re2s with Scala 2.12.6. We assured for all benchmarks that all
scalac methods were JIT compiled and that no GC was performed during the benchmark runs. The benchmarks are executed in a controlled environment with all input and output in memory. The following graph shows speedups of GraalVM CE and GraalVM EE compared to the Java HotSpot VM when compiling different Scala applications.
In practice, Scala code is compiled with the SBT build tool. To evaluate compilation speedup that users can expect in a real setup, we checked out the following projects: Shapeless, the Scala compiler, and Akka. For each project, we entered the SBT shell and executed a series of
;clean ;compile commands until the compilation time was stabilized. Then, we measured the time it takes for the last compilation. For this benchmark, we keep all input and output data on a fast SSD. The following graph shows that GraalVM performs better than the Java HotSpot VM by 1.3x.
GraalVM CE (Community Edition) is open source. GraalVM EE (Enterprise Edition) can be downloaded and used for evaluation purposes. Enabling GraalVM in SBT is as easy as
sbt --java-home <path-to-graal-vm>
Building a Native Image of the Scala Compiler
The Scala compiler also has slow startup times. If we take the Scala compiler and run it on a “Hello, World!” program we get the following
$ time scalac HelloWorld.scala real 0m1.866s user 0m6.549s sys 0m0.259s
GraalVM comes with a tool for building native executables from programs written in JVM-based languages (Java, Scala, Kotlin, …):
native-image. It performs a points-to analysis to determine reachable types, fields, and methods given an entry point to the program. During the points-to analysis,
native-image substitutes reachable methods with restrictions and takes a snapshot of all the state reachable from initialized static fields. Then,
native-image uses the Graal compiler to compile reachable methods to native code for the given platform and links this code with the snapshot of the static state.
native-image can produce both executables and shared libraries. The native image generation is currently supported on Java 8 on Linux and Darwin. Support for newer versions of Java as well as Windows support is in the works.
Now let’s try to build a native image of the Scala compiler with the
native-image tool. For that, we need to check out and compile substitutions for a few methods in the compiler:
git clone https://github.com/graalvm/graalvm-demos cd graalvm-demos/scala-days-2018/scalac-native/scala-substitutions sbt package cd ../
Now we can build a native image of the Scala compiler with
scalac-image.sh script puts all Scala classes on the classpath, links substitutions that are necessary for the compiler, and declares classes that the Scala compiler loads reflectively. This script requires that the environment has
GRAALVM_HOME properly set. When the image build is complete, we can invoke the compiler with the provided wrapper script (
scalac-native ) on the given “Hello, World!” example:
time ./scalac-native HelloWorld.scala real 0m0.177s user 0m0.129s sys 0m0.034s
and see a 10x improvement on the cold start. Here’s how the startup speedup looks like for the Java HotSpot VM and a native image of
Support for Macros
Since macros are dynamically loaded, we can not build a native image that will support all macros in the world. For macros to work the macro classes must be known to the image builder of the Scala compiler. To try a
scalac image that includes macros we can run
./scalac-native macros/GreetingMacros.scala -d macros/./scalac-image-macros.sh
scalac-image-macros.sh script adds the list of macros to the set of reflectively accessed elements.
Now, we can compile a project that uses macros from
./scalac-native -cp macros/ HelloMacros.scala
and execute the compiled program with
$ scala HelloMacros Hello, World!
GraalVM allows to compile Scala applications ~1.3x faster. You can get the GraalVM binaries and evaluate it on your compilation and enjoy the productivity boost.
If you are more interested in instant startup, you can build a native image of
scalac and other projects. Native image also has lower memory footprint than running on a full Java VM.