When in Doubt: Reflection

We want to kick the new year off with a major update and start a series of blog posts about the features we introduced. Also, kittens.

Writing code has always been about APIs that are easy to use and understand. Code is not written for machines, it is written for humans.
Reflection is often used to hide implementation details, reduce boilerplate code or to perform dependency injection. It is an important part of Java.

Version 0.2.40 introduces many new APIs and full reflection support is one of them. Obviously this comes at a cost. Both code size and performance need to be evaluated. In this post we want to explain how reflection works and how we keep your code size from exploding.


Minifying and the Keep-Set

Reflection allows you to access classes at runtime that you don’t reference in your code. It means that every class, every method and every field needs to be accessible at runtime. This is of course far from being feasible when compiling from the web since you’d end up with a huge JavaScript file.

The defrac compiler is always minifying your code so that only the actual classes in use are being translated to JavaScript. This is obviously problematic when using reflection since classes, methods and fields can get lost if you don’t reference them directly. If you’ve used Proguard before, you’ve probably experienced a similar problem.

Example

We have a class, Point, that has two fields. But only the x-Field is accessed. Therefore the defrac compiler wouldn’t include the y-Field in the generated output.

class Point {
public double x;
public double y;
}
Point p = new Point();
p.x = 1.0;

Since we don’t include the y-Field, reflection wouldn’t find it when compiling with defrac. The following code would end up throwing a NoSuchFieldException .

assert p.getClass().getDeclaredField("y").getDouble(p) == 0.0;

There are several options to solve this problem.

Auto-Includes

If we can deduce that you are using a class, method or field at compile time, we will include it. This happens currently only for literals. Java knows many different kinds of literals. true is a boolean-literal, “Hello World” is a string-literal and Point.class is a class-literal.

The compiler will automatically include classes, methods and fields when one of the following forms is used:

  • Foo.class.getDeclaredField(“foo”)
    Include the field foo of Foo
  • Foo.class.getField(“foo”)
    Include the field foo of Foo
  • Foo.class.getMethod(“foo”)
    Include the method foo of Foo
  • Foo.class.getDeclaredMethod(“foo”, int.class)
    Include the method foo(int) of Foo
  • Foo.class.newInstance()
    Include Foo’s default constructor
  • Foo.class.getDeclaredConstructor(int.class)
    Include the Foo(int) constructor
  • Class.forName(“Foo”)
    Include the Foo class and its constructors

Note that you have to use literals here. Otherwise the compiler won’t know ahead of time what’s happening.

In order to make the Point example work, we could simply use the following code:

assert Point.class.getDeclaredField("y").getDouble(p) == 0.0;

We simply replaced p.getClass() with Point.class and now the compiler knows that we want to access the y-Field although it has not been referenced in the code.

The Keep-Set

It is not always possible to use literals throughout the code. Therefore we introduced the keep-set.

The keep-set allows you to specify individual classes or members that have to bee considered when compiling. It is also possible to include all classes from within a package or a package and its sub-packages. It is part of the project configuration and can be configured using the config command.

  • config keep +foo.bar.Baz
    Keep the Baz class and all its members
  • config keep +foo.bar.Point#y
    Keep the y-Field or y-Method(s) of Point
  • config keep +foo.bar.*
    Keep all classes and their members in the foo.bar package
  • config keep +foo.bar.**/*
    Keep all classes and their members in the foo.bar package and all its sub-packages

The rationale is quite simple. Reflection is often used within distinct packages of your application, like your serialization logic. In that case it is quite simple to add the corresponding package to the keep-set and continue coding without having to worry about missing fields or classes.

What If I Don’t Use Reflection?

We make every effort to keep code size to a minimum while preserving correctness. When using reflection, we have to generate a lot more information to satisfy the runtime. But what if you don’t use reflection?

The answer is simple: Nothing happens.

You won’t see your code size grow since we don’t generate all this information if reflection is not used at all. Therefore it make sense to opt-out of reflection if you can. Your code-size will be smaller and it is also more memory friendly.

Serialization

Full reflection support allows us to implement other features like serialization. We implemented it in an efficient manner that is also friendly to your code size and consistent across different compiles.

The Ominous serialVersionUID

Serialization internals are usually not of importance. But when omitting fields of a class, it can become quite crucial to understand them.

The serialVersionUID field —which has to be declared private, static and final — is part of the serialization contract. It defines the version (or shape) of a class at the time of serialization. If you don’t define a serialVersionUID, one is generated for you using an algorithm defined in the Serialization Specification. It basically means that your class will get a different serialVersionUID depending on the fields it declares. You can already guess what’s the problem here. defrac gets rid of unused fields and methods.

In order to keep things snappy and consistent, the compiler will inject a serialVersionUID field into each serializable class, if not already existing. This serialVersionUID is based on the actual class, as if no field or method would be omitted
(Bonus-Nerd-Fact: the compiler omits even an interface in the type signature if proven to be unnecessary). This means that the Point class from above would always have the same serialVersionUID, no matter if only one of its field is being accessed and the other one omitted.

Compatibility with other JVMs

If an object has a serialVersionUID that matches its shape but doesn’t write all fields to the output stream it would lead to other problems since values go missing.

There are two options to solve this issue. The compiler could basically synthesize a readObject and writeObject method for each serializable class. This would yield better performance but introduce a lot of code. We believe that this is not the right way to go forward at this stage and might introduce this in the future as some kind of advanced optimization option.

Instead each serializable class will include all its serializable fields by default. This makes it possible to go with a single default implementation that iterates through all the declared fields.

We successfully tested the compatibility of output generated by defrac against HotSpot, OpenJDK and Dalvik. You can use defrac now on the client to generate an object, serialize it and send it to your server JVM without any special configuration.

What If I Don’t Use Serialization?

Like with reflection we’ll make sure to generate serialVersionUID only if serialization is actually being used.

The compiler won’t perform any of the mentioned transformations if you don’t use ObjectOutputStream.writeObject or ObjectInputStream.readObject.

Wrap Up

Reflection and serialization are two powerful new tools. They are already available when compiling for Android, the JVM or the Web. We are currently working on iOS support and will release it in the next weeks.

Thanks to reflection and a beefed-up runtime, it’s now possible to use excellent libraries like Google GSON.

You can always make sure you’re using the latest defrac version by running:

defrac --force-update
Show your support

Clapping shows how much you appreciated defrac’s story.