Is Kotlin default arguments a solution of binary backward compatibility?

TL;DR,

Adding arguments to existing API will break binary backward compatibility, even with Kotlin default value. Using @JvmOverloads can be a workaround in some case, but it’s not a perfect solution.

Introduction of the issue

Let’s imagine a scenario which happens in our daily development life: we have a functional library which is depended by both the application and its feature library.

And we have this public class which is both used by the feature library and application, and let’s say it has version number of 1.0.0:

class Request(val id: String, val url: String)

One day, in the functional library we need to add a property in this class to allow the Request to carry some request parameters. With first thought, here is the modification we are going to have and we bump the functional version to 2.0.0:

class Request(val id: String, val url: String, val params: Params)

As known by all of us, this adding of argument of params will break the public API, and the best way is that we bump the version of the function library in the feature library from 1.0.0 to 2.0.0, recompile, build and publish it, and finally bump both libs in the application.

But what if you have hundreds of feature libraries? Or what if the functional library which you are developing is a third-party SDK which is used by amount of other developers? It becomes unrealistic to let them bump the functional library as you want, so a binary backward compatible solution is what we are looking for.

Is Kotlin Default argument backward compatible?

With Kotlin, we can have a default value for the parameter, so with the second thought, we are probably going to do this:

class Request(val id: String,
              val url: String,
              val params : Params = Params()
)

We are expecting that the default value can be provided when the old API is called without params argument, so we should be exempted to release a new feature library version

//Inisde application, we use functional library with version 2.0.0
val request = Request(id = "app_request", 
                      url = "upload/my_request", 
                      params = Params("user", "john")
)

//Inside feature library, we use functional library with version 1.0.0
val request = Request(id = "feature_lib_request", 
                      url = "upload/my_request"
)

Let’s compile the app and run it, oops, there is a crash saying:

java.lang.NoSuchMethodError: 
'void Request.<init>(java.lang.String, java.lang.String)'

What happens? why the feature library cannot keep calling the old constructor and benefit from the default value of the new argument?

How default argument work?

First thing that we need to know is that, since both app and feature library depend on the functional library, they cannot have two different version when finally app is compiled, the old version 1.0.0 from feature library will be override by 2.0.0 from app since the app’s version is higher.

Since the new version of Request class is taken into account, we need to decompile its Kotlin source code into java file to understand how the default argument work in java:

public Request(@NotNull String id, @NotNull String url, @NotNull Param param) {
   ...
}

// $FF: synthetic method
public Request(String var1, String var2, Param var3, int var4, DefaultConstructorMarker var5) {
   ...
}

Apparently, since the feature library is never recompiled with the version 2.0.0 of functional library, the only constructor signature it knows is `void Request.<init>(java.lang.String, java.lang.String)`, only when you recompile the feature library with 2.0.0 version, it can take into account the created synthetic method to handle the default value.

Will @JvmOverloads Save us?

As a workaround, we may add @JvmOverloads on the constructor, will it work? let’s try:

class Request @JvmOverloads constructor(
      val id: String,
      val url: String,
      val param : Param = Param()
)

And we try to recompile only the functional library, then run the application again, it doesn’t crash anymore! And if we decompile the Kotlin file this time, we can see a new constructor added in the java file:

public Request(@NotNull String id, @NotNull String url, @NotNull Param param) {
   ...
}

// $FF: synthetic method
public Request(String var1, String var2, Param var3, int var4, DefaultConstructorMarker var5) {
   ...
}

@JvmOverloads
public Request(@NotNull String id, @NotNull String url) {
  ...
}

Thanks to this @JvmOverloads function, the feature library can match the signature of the constructor and instantiate Request class even if it keeps the old execution. Wonderful until now, so may we consider that @JvmOverloads can prevent from the incompatibility in all the case? Of course NOT!

How @JvmOverloads works?

Let’s expand our example a little bit, assuming that we accept the solution of @JvmOverloads on our constructor, and with the growth of our business, we have more and more fields in our Request class:

class Request @JvmOverloads constructor(
    val id: String,
    val url: String,
    val timeout : Long = 30,
    val shouldRetry: Boolean = false,
    val param : Param = Param()
)

Different feature libraries may instantiate the class in different ways:

//feature library A, skipping `shouldRetry` argument
val requestA = Request(id = "feature_request_A", 
                      url = "upload/my_request", 
                      timeout = 60,
                      params = Params("user", "john")
)


//feature library B, skipping `shouldRetry` and `param` argument
val requestB = Request(id = "feature_request_B", 
                      url = "upload/my_request", 
                      timeout = 120,
)

//feature library C, not skipping anything
val requestC = Request(id = "feature_request_C", 
                      url = "upload/my_request", 
                      timeout = 60,
                      shouldRetry = true,
                      params = Params("user", "john")
)

Of course, business never stops growing, we have to add another argument inside Request :

class Request @JvmOverloads constructor(
    val id: String,
    val url: String,
    val timeout : Long = 30,
    val shouldRetry: Boolean = false,
    val param : Param = Param(),
    val encrypted : Boolean = true, //new optional argument
)

Once we bump the function library inside app, we found the app crashes again! And this time the stack trace points to the feature library A and feature library B. So it’s the time for the decompling of the source code again, and this time we see the @JvmOverloads created following constructor for us:

@JvmOverloads
public Request(@NotNull String id, @NotNull String url, long timeout, boolean shouldRetry, @NotNull Param param, boolean encrypted) {
   ...
}

@JvmOverloads
public Request(@NotNull String id, @NotNull String url, long timeout, boolean shouldRetry, @NotNull Param param) {
  this(id, url, timeout, shouldRetry, param, false, 32, (DefaultConstructorMarker)null);
}

@JvmOverloads
public Request(@NotNull String id, @NotNull String url, long timeout, boolean shouldRetry) {
  this(id, url, timeout, shouldRetry, (Param)null, false, 48, (DefaultConstructorMarker)null);
}

@JvmOverloads
public Request(@NotNull String id, @NotNull String url, long timeout) {
  this(id, url, timeout, false, (Param)null, false, 56, (DefaultConstructorMarker)null);
}

@JvmOverloads
public Request(@NotNull String id, @NotNull String url) {
  this(id, url, 0L, false, (Param)null, false, 60, (DefaultConstructorMarker)null);
}

so let’s make a table about the presence of all the optional arguments in each constructor:

If we do a simple math about all the combinations of presence of argumnts, we get 2⁵ = 32 possibilities. So clearly there are stills some constructors missing which @JvmOverloads doesn’t create for us, and that’s why feature library A and B can not find the matching constructor for their executions.

In the documentation of @JvmOverloads annotation we can also see the explanation of how it works:

Instructs the Kotlin compiler to generate overloads for this function that substitute default parameter values.

If a method has N parameters and M of which have default values, M overloads are generated: the first one takes N-1 parameters (all but the last one that takes a default value), the second takes N-2 parameters, and so on.

But why @JvmOverloads doesn’t create all for us?

Let’s take a glance in the APK after building the application, inside classes.dex file, we can find the size of each constructor, imagine if one day the Request class have 10 optional arguments (which is not quite rare in the real development) with default values in the constructor, and if the annotation creates all the constructors for us, then it will be totally 2¹⁰ = 1024 constructors, which can take more than 100KB storage just for this class, which leads our application quite huge.

So as the conclusion, @JvmOverloads will prevent the inconsistent of API if the previous execution of functions have all the optional argument present (as shown in feature library C), otherwise if we skip any optional argument in the previous compilation, adding a new option argument in the functional library will cause the inconsistency, which can lead crash.

Reference

Backward compatibility

JvmOverloads