Kotlin Native Interop Generics

This is a longer explanation of the Objc implementation for generics with Kotlin Native Interop framework generation.

To interop with Kotlin Native from an iOS/Macos client (from now on, “Darwin client”), you can specify that konan generate an Xcode framework. That includes binary code, and an Objc header. We’ve made modifications to that header to support generics output.

Goal

Make generic information available for user-defined (ie. not built in) Kotlin classes, on a best efforts basis, to facilitate Darwin development using Kotlin libraries. Generics will be output in Objc, but this is primarily desired to improve the Swift dev experience.

Visually, something like this:

General Guidelines

We’re operating under the assumption that most users want generic support to facilitate consuming a Kotlin API. Generics will make results from calls to APIs more readable and discoverable. As Objc generics are “light” and don’t really mean anything at runtime, the inclusion of them is simply to help with tooling and to avoid a lot of casting.

The goal is to help the usual case, but obviously not to create a situation where the developer can’t cast around an incompatibility. This will come up mostly with variance. As it currently stands, the usual case with a lack of generics means an Objc/Swift dev has to cast on all interactions. Our goal is to move casting into edge cases.

TL;DR Make API’s more readable, make casting edge case, but don’t paint yourself into a corner.

3 Languages

Kotlin, Objc, and Swift are fundamentally three different languages. Kotlin and Objc are obviously different, and there is detail that will be lost in transit. Kotlin and Swift are conceptually similar, but differ a lot in implementation. Any effort to interop between them will require compromises, and those compromises will be larger in scope as the feature in question is more complex.

Put another way, an Int in Kotlin doesn’t have to do much to be an Int32 in Swift. Kotlin can’t represent Swift value struct types directly at all (nor can Objc). Generics will fall somewhere in the middle, and we think what those compromises should be is a discussion (and why you’re reading this).

Compatibility Issues

Variance

The Kotlin docs on variance are a great introduction to variance as a topic. Going deeper into that topic is a bit out of scope for this post, but please check out that doc for background.

In general, Kotlin supports the most in terms of features and Variance. Objc supports declaration-site variance (not use-site AFAIK). Swift supports no variance on user defined types, although built-in types have some (Optionals, Collections, Functions). Neither Objc nor Swift support generics on protocols, btw (which are interfaces in Kotlin/Java).

The basic choice when outputting a class with a variant generic definition is we either output it as best we can, or ignore it.

Since we are outputting Objc, and Objc supports generics on classes, we are simply outputting _covariant/_contravariant where defined. In this case, because Swift has to deal with this now, Swift will be able to cast around any incompatibilities.

That means this Kotlin:

class GenVarOut<out T>()

Will (roughly) produce this Objc

@interface MainGenVarOut<__covariant T>

And Swift will let you do this (worst case)

let a : GenVarOut<ParentType> = GenVarOut<ChildType>(b) as! GenVarOut<ParentType>

That’s not exactly pretty, but again the point is in the normal case, you’re doing less casting, but in edge cases you aren’t stuck.

Use-site variance isn’t supported. A star type currently turns into ‘id’, which roughly translates to AnyObject?.

//Kotlin
fun starGeneric(arg: GenNonNull<*>)
    
//Objc
+ (void)starGenericArg:(MainGenNonNull<id> *)arg __attribute__((swift_name("starGeneric(arg:)")));
    
//Swift
func starGeneric(arg: GenNonNull<AnyObject>)

We’re also under the impression you can cast around anything here, but to discuss.

Nullability

Kotlin and Swift define nullability on types. Syntactically speaking, SomeData vs SomeData?. Objc is different. Nullability is defined on the properties and methods of a class.

For generics, that means an unbounded generic parameter will wind up with nullable methods and property definitions in Objc (and ultimately Swift).

Notice the nullable types in Swift on the right. That is logical Swift. It’s really coming from Objc, which looks like this:

Because nullability is on the methods of a class, if it’s possible that a value can be null, the type must always be nullable. That is a consequence of using Objc for interop.

However, there is a simple mitigation, assuming you are defining your generic code. Make the generic parameter’s upper bound non-null.

class Cage<T: Any>

That will allow the generated Objc to mark the methods and properties as nonnull.

Constraints

All three languages let you define upper bounds on generics. However, what those languages support differs. Kotlin specifically supports some self-referencing, recursive-ish, definitions. You’ll run into this right out of the gate.

public abstract class Enum<E : Enum<E>>

Objc does not digest the equivalent. We decided to omit upper bounds for now. It would be possible to add a partial implementation, but since we think most developers will simply be consuming API’s, and since we wanted to get the discussion started, in the interest of time we simply dropped them for now.

As mentioned, however, if your Kotlin code defines a non-null constraint on a type, Objc will have non-null method/property declarations, and we highly encourage doing so where possible.

Direct Swift Interop

I wanted to sidebar about the possibility of direct Swift interop with Kotlin. There has been a general belief in the Kotlin Native community that the only major hurdle to supporting Swift interop was ABI stability, and now that that’s emerging, most Interop issues will go away. That is really not true.

Kotlin and Swift share many conceptual capabilities that are implemented in incompatible ways. Swift Value types, and structs specifically, are very popular and totally not supported in Kotlin. They conceptually fill a similar role that data class fill in Kotlin, but they’re different, and ABI stability will do nothing to solve that on it’s own. Same situation with “enums with associated values” and sealed classes.

For generics specifically, Swift is a lot more strict with other Swift than with Objc. The following will work if the generic is defined in Objc

let a : GenVarOut<ParentType> = GenVarOut<ChildType>(b) as! GenVarOut<ParentType>

That would fail at runtime with Swift. Crash fail. Whether Swift will treat Swift interop code the same as actual Swift code remains to be seen, but I assume it will.

Some things will certainly improve with direct Swift interop, but it is by no means better or simpler across the board.

With regards to our choices with Objc output and generics, one argument to avoid doing this is if direct Swift interop happens in the future, some of our Objc interop choices, variance for example, will force incompatible source changes if the developers want to move to direct Swift interop.

Our thinking here is the following:

1. We don’t know if/when Swift interop will happen
2. Objc interop won’t go away
3. Any team making the switch will have to make a deliberate decision, and generics won’t be the only consideration, nor will generics likely be the only required source change downstream

Overall, this PR is meant to start a discussion, and that should certainly be a part of it.

Implementation

Objc interop is obviously a core part of Kotlin Native, which presents some difficulty in getting this significant of a change into the repo. Generics have been added as an experimental flag, and hopefully we can get this feature into a release build for devs to play with.

Generics are disabled by default. To enable them, send extraOpts "-Xobjc-generics"to the component config. For example:

components.main {
    outputKinds("framework")
    extraOpts "-Xobjc-generics"
}

If not published to a production build, you can clone/build Kotlin Native locally, and point to that build in your project’s gradle.properties

org.jetbrains.kotlin.native.home=[your KN dir]/dist

Kotlin native builds will need to correspond to published versions if you’re using libraries. When 1.3.30 is released, I’ll make sure there’s a fork branch that can be built and will be compatible with that release.

I’m submitting a draft PR to Kotlin Native master, but you can follow the changes and release versions in my fork.