Creating CLI tools with Scala, Picocli and GraalVM

Previously, I wrote an article about how to create CLI tools with Scala. As I wrote in the article, writing CLI tools with Scala has various advantages for me as a Scala programmer.

However, there is a disadvantage that is the overhead of launching JVM. I was wondering if we can create a native image of my Scala-based CLI tools using GraalVM. Recently, I found Picocli which is a framework for Java-based CLI applications with native-image creation supports by GraalVM.

remkop/picocli

Immediately, I started trying to use Picocli with Scala.

Create SBT project

I tried to port checksum command which is explained in the documentation of picocli to Scala. Firstly, I created a simple Picocli available project.

name := "picocli-scala-example"
version := "0.1"
scalaVersion := "2.12.7"
libraryDependencies ++= Seq(
  "info.picocli" % "picocli" % "4.2.0"
)

Source code of the checksum command itself:

package com.github.takezoe
import java.io.File
import picocli.CommandLine._
import java.math.BigInteger
import java.nio.file.Files
import java.security.MessageDigest
import java.util.concurrent.Callable
import picocli.CommandLine
@Command(
  name = "checksum", 
  mixinStandardHelpOptions = true, 
  version = Array("checksum 4.0"),
  description = Array(
    "Prints the checksum (MD5 by default) of a file to STDOUT."
  )
)
class CheckSum extends Callable[Int] {
@Parameters(
    index = "0", 
    description = Array("The file whose checksum to calculate.")
  )
  private var file: File = null
@Option(
    names = Array("-a", "--algorithm"), 
    description = Array("MD5, SHA-1, SHA-256, ...")
  )
  private var algorithm = "MD5"
override def call(): Int = {
    val fileContents = Files.readAllBytes(file.toPath)
    val digest = MessageDigest
      .getInstance(algorithm)
      .digest(fileContents)
    printf("%0" + (digest.length * 2) + "x%n", 
      new BigInteger(1, digest))
    0
  }
}
object CheckSum extends App {
  val exitCode = new CommandLine(new CheckSum()).execute(args:_*)
  System.exit(exitCode)
}

As you can see, this is a normal Scala program except that we have to define fields as var and be annotated by picocli’s annotations.

Run annotation processor

Picocli offers code generation by annotation processor for GraalVM native-image support. This means we need to run the annotation processor on SBT before creating native-image.

The annotation processor is included in picoli-codegen. We need to add it to the library dependencies as provided scope.

libraryDependencies ++= Seq(
  "info.picocli" % "picocli" % "4.2.0",
  // Add a line below
  "info.picocli" % "picocli-codegen" % "4.2.0" % "provided"
)

Also, I added the following configuration to build.sbt to run the annotation processor as processAnnotation task.

lazy val processAnnotations = taskKey[Unit]("Process annotations")
processAnnotations := {
  val log = streams.value.log
  log.info("Processing annotations ...")
val classpath = ((products in Compile).value ++ ((dependencyClasspath in Compile).value.files)) mkString ":"
  val destinationDirectory = (classDirectory in Compile).value
  val processor = "picocli.codegen.aot.graalvm.processor.NativeImageConfigGeneratorProcessor"
  val classesToProcess = Seq("com.github.takezoe.CheckSum") mkString " "
val command = s"javac -cp $classpath -proc:only -processor $processor -XprintRounds -d $destinationDirectory $classesToProcess"
runCommand(command, "Failed to process annotations.", log)
log.info("Done processing annotations.")
}
def runCommand(command: String, message: => String, log: Logger) = {
  import scala.sys.process._
  val result = command !
if (result != 0) {
    log.error(message)
    sys.error("Failed running command: " + command)
  }
}

Install GraalVM

To create a native image, we, of course, need to install GraalVM. We can download GraalVM Community Edition from GitHub.

graalvm/graalvm-ce-builds

Extract tar.gz and add bin directory to PATH. Then install native-image command using gu command as follows:

$ gu install native-image

Create native image

I used sbt-native-packager to create a native-image on SBT.

sbt/sbt-native-packager

Added a line below to project/plugins.sbt:

addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.6.1")

Also, added the following configuration to enable native-image creation with annotation processing defined in advance:

packageBin in Compile := (packageBin in Compile dependsOn (processAnnotations in Compile)).value
enablePlugins(GraalVMNativeImagePlugin)

Everything is ready. Now we can generate a native image by running SBT as follows:

$ sbt graalvm-native-image:packageBin

A binary is generated under target/graalvm-native-image with the project name. Let’s run the generated binary.

$ ./target/graalvm-native-image/picocli-scala-example
Missing required parameter: <file>
Usage: checksum [-hV] [-a=<algorithm>] <file>
Prints the checksum (MD5 by default) of a file to STDOUT.
      <file>      The file whose checksum to calculate.
  -a, --algorithm=<algorithm>
                  MD5, SHA-1, SHA-256, ...
  -h, --help      Show this help message and exit.
  -V, --version   Print version information and exit.

It worked!

Conclusion

We can create a native image of Scala-based CLI tools with Picocli and GraalVM. Of course, as you may know, there are several restrictions in native-image by GraalVM. For example, the use of reflection is restricted. However, if you want to create CLI tools which don’t have the overhead of launching JVM, it would be a promising solution.

By the way, there is another solution, scala-native in the Scala world. However, scala-native faces a lack of development resource. Actually, it still stays at Scala 2.11. Also, few libraries support scala-native. I think GraalVM is a better solution for creating native-image in Scala at this moment.

A project I created through this article is below:

takezoe/picocli-scala-example