Shrinking Kotlin libraries and applications using Kotlin reflection with R8

Co-authored by Morten Krogh-Jespersen and Mads Ager

R8 is the default application shrinker for Android. R8 reduces the size of Android applications by removing unused code and optimizing the code that remains. R8 also has support for shrinking Android libraries. In addition to producing smaller libraries, library shrinking can also be useful to hide new features of your library until you are ready to release them or talk about them publicly.

Kotlin is a great language for writing Android applications and libraries. However, shrinking Kotlin libraries or applications that use Kotlin reflection comes with some challenges. Kotlin uses metadata in Java class files to identify Kotlin language constructs. If your application shrinker does not maintain and update Kotlin metadata, your library or application will not work.

R8 now has support for maintaining and rewriting Kotlin metadata to fully support shrinking of Kotlin libraries and applications using Kotlin reflection. The support is available in the Android Gradle Plugin version 4.1.0-beta03. Please try it out and let us know how it works for you and file any issues that you encounter in our public bug tracker.

The rest of this blog post provides information on Kotlin metadata and on R8’s support for Kotlin metadata rewriting.

Kotlin metadata

Kotlin metadata is extra information stored in annotations in Java class files produced by the Kotlin JVM compiler. This metadata specifies which Kotlin language construct a given class or method in the class file corresponds to. For example, Kotlin metadata is what allows the Kotlin compiler to recognize that a method in a class file is actually a Kotlin extension function.

Let’s have a look at a simple example. The following library code defines a hypothetical base command builder for building compiler commands.

We can then define a concrete, hypothetical D8CommandBuilder on top of CommandBuilderBase for building a simplified D8 command.

The example uses extension functions in order to make sure that if you call the setMinApi method on an D8CommandBuilder the type of the object returned will be D8CommandBuilder and not CommandBuilderBase. These extension functions are top level and they live on the file class CommandBuilderKt for our example. Let’s have a look at that class file using (simplified) javap output.

$ javap com/example/mylibrary/CommandBuilderKt.class
Compiled from "CommandBuilder.kt"
public final class CommandBuilderKt {
public static final <T extends CommandBuilderBase> T addInput(T,      String);
public static final <T extends CommandBuilderBase> T setMinApi(T, int);
...
}

The javap output shows that the extension functions compile to static methods that take an extra first parameter which is the extension receiver. This information is not enough for the Kotlin compiler to understand that these methods can be used from Kotlin code as extension functions. Therefore, the Kotlin compiler also puts a kotlin.Metadata annotation in the class file. This annotation contains metadata with extra Kotlin-specific information for this class. The annotation shows up if we use the verbose flag withjavap.

$ javap -v com/example/mylibrary/CommandBuilderKt.class
...
RuntimeVisibleAnnotations:
  0: kotlin/Metadata(
   mv=[...],
   bv=[...],
   k=...,
   xi=...,
   d1=["^@.\n^B^H^B\n^B^X^B\n^@\n^B^P^N\n^B...^D"],
   d2=["setMinApi", ...])

The d1 field of the metadata annotation contains most of the actual metadata in the form of a protocol buffer message. The precise contents of the metadata is not important. The important thing is that the Kotlin compiler reads this metadata and uses it to figure out that these methods are extension functions, as illustrated by the following kotlinp dump.

$ kotlinp com/example/mylibrary/CommandBuilderKt.class
package {
// signature:   addInput(CommandBuilderBase,String)CommandBuilderBase
public final fun <T : CommandBuilderBase> T.addInput(input: kotlin/String): T
// signature: setMinApi(CommandBuilderBase,I)CommandBuilderBase
public final fun <T : CommandBuilderBase> T.setMinApi(api: kotlin/Int): T
...
}

This metadata allows the use of these functions as Kotlin extension functions in Kotlin user code:

How R8 used to break Kotlin libraries

As the previous section shows, the Kotlin metadata for the class files in a library is really important to be able to use the Kotlin API of the library. However, the metadata is in an annotation and encoded as a protocol buffer message that R8 used to know nothing about. Therefore, R8 used to do one of two things:

1. Throw away the metadata.
2. Keep the original metadata.

Both of these options are bad.

If the metadata is thrown away, the Kotlin compiler will no longer understand that extension functions are extension functions. Therefore, for our example, when compiling code such as D8CommandBuilder().setMinApi(12) the compiler will produce an error stating that no such method exists. That makes sense, because without the metadata, the only thing the Kotlin compiler can see is a Java static method with two parameters.

Keeping the original metadata is bad as well. One of the things that is recorded in Kotlin metadata is super types for classes. So, suppose we only want the D8CommandBuilder class to keep their names when shrinking the library. That would mean that CommandBuilderBase would be renamed, most likely to a. If we leave the original Kotlin metadata, the Kotlin compiler will look for the super type of D8CommandBuilder recorded in the Kotlin metadata. If we use the original metadata, the recorded supertype is CommandBuilderBase and not a. The compilation will therefore fail with an error stating that the super type CommandBuilderBase does not exist.

R8 Kotlin metadata rewriting

In order to fix these issues, R8 has been extended with the capability to maintain and rewrite Kotlin metadata. This is done by embedding the Kotlin metadata library developed by JetBrains in R8. The metadata library is used to read the Kotlin metadata in the original input. The metadata information is recorded in R8’s internal data structures. When R8 is done optimizing and shrinking the library or application, it synthesizes new correct Kotlin metadata for all of the Kotlin classes that are explicitly kept.

Let’s have a look at what that looks like for our example. We put the example code into an Android Studio library project. We enable shrinking by setting minifyEnabled to true as usual in the gradle build file. We update the shrinker configuration to contain the following.

This tells R8 to keep D8CommandBuilder and all of the extension functions on CommandBuilderKt. It also tells R8 to keep annotations and in particular to keep the kotlin.Metadata annotation. These rules will only keep Kotlin metadata on the classes that are explicitly kept. Therefore, this will maintain the metadata on D8CommandBuilder and CommandBuilderKt, but not on CommandBuilderBase. We are doing it this way to make sure that you are not shipping a lot of useless metadata in your applications and libraries.

Now, building the library with shrinking enabled produces a library in which CommandBuilderBase has been renamed to a. Additionally, the Kotlin metadata for all the kept classes has been rewritten so that any reference to CommandBuilderBase now refers to a. This makes the library work as a Kotlin library as expected.

As a final note, not keeping the Kotlin metadata on CommandBuilderBase means that the Kotlin compiler will treat the resulting class in the output as a Java class. That can lead to strange completions in your library for Java implementation details of the Kotlin class. To avoid that it is possible to keep the class. If we do, metadata will be maintained. We can use the allowobfuscation modifier on the keep rule to allow R8 to rename the class, while also generating Kotlin metadata for it so the Kotlin compiler and Android Studio will see the class as a Kotlin class.

So far, we have been talking about library shrinking and how Kotlin metadata is needed for Kotlin libraries. Kotlin metadata is also needed for applications that use Kotlin reflection via the kotlin-reflect library. The issues are exactly the same as for libraries. If Kotlin metadata is removed or is not updated correctly, the kotlin-reflect library will not be able to understand the code as Kotlin code.

As a simple example, say we want to look up and call an extension function on a class at runtime. We want to allow the method to be renamed as we do not care about the name, we just want to find it and call it at runtime.

In our code, we add a call: reflect(ReflectOnMe()). This will find the extension function defined on ReflectOnMe and call it using the given ReflectOnMe instance as the receiver and the string “reflection” as the extension receiver.

Now that R8 correctly rewrites Kotlin metadata on all kept classes, we can make this work when shrinking the app using the following shrinker configuration.

This allows both ReflectOnMe and extension to be renamed, it maintains and rewrites the Kotlin Metadata and the app works.

Give it a try!

Please try R8’s support for Kotlin metadata rewriting on Kotlin library projects as well as on Kotlin projects using Kotlin reflection. The support is available in the Android Gradle Plugin starting in version 4.1.0-beta03. If you encounter any issues using it, please file bug reports in our public bug tracker.

Happy shrinking!