Generated JSON-Serialisation for Kotlin
For quite some time I used Gson for serialisation in our Kotlin and Java-applications. It was the most brittle part of our code, for several reasons:
- When Gson doesn’t find a no-arg constructor, sun.misc.Unsafe is used. This means initialisers aren’t called and the deserialised objects could be broken.
- Gson uses reflection to set and retrieve fields. Reflection is slow on Android.
- Nulls in JSON are able to break Kotlin’s null-safe typing.
To improve on that situation, I decided to write an Annotation Processor that generates serialisation code at compile-time, for that would give the most freedom. Here are my design-choices:
All necessary data needs to be in the constructor
I decided to instantiate deserialised objects with an existing constructor only (the one with the most arguments). This ensures that they are always initialised properly.
The only needed annotation is @GsonSerializer
which goes on the top of a class:
@GsonSerializer
class AppState(val userName: String,
val notifications: List<String>)
This forces separation of initial from generated data. It also removes the need to exclude lazy
or @Transient
properties.
The generated serialiser is an extension function on JsonWriter
:
fun JsonWriter.value(obj: AppState?) {
if (obj == null) {
nullValue()
return
}
beginObject()
name("userName").value(obj.userName)
name("notifications").array(obj.notifications) { value(it) }
endObject()
}
The field names and types are taken from the constructor. The code assumes they are also accessible as members, which is easy to obtain.
Developer-friendly deserialisation
Deserialisation is more complex, since it does a couple of things to help development:
- It throws informative errors when fields are missing
- Field-names and enum-values are matched case-insensitive. I realise this is debatable, but it prevents some flow-breaking bugs and was never an issue in production. Serialisation keeps the original casing.
The deserialiser:
fun JsonReader.nextAppState(): AppState { var userName: String? = null
var notifications: MutableList<String>? = null beginObject()
while (hasNext()) {
when (nextName().toLowerCase()) {
"username" -> userName = nextString()
"notifications" -> notifications = readList(JsonReader::nextString)
else -> skipValue()
}
}
endObject() val obj = AppState(
userName = checkNotNull(userName) {
"error parsing AppState, field not found: userName: String"},
notifications = checkNotNull(notifications) {
"error parsing AppState, field not found: notifications: List"}
) return obj
}
Nested objects
Since all methods are created as extensions on JsonWriter
and JsonReader
adding support for more fields is as simple as creating a new extension method or adding @GsonSerializer
to the referenced class:
@GsonSerializer
class AppState(val user: UserInfo,
val time: ZonedDateTime)class UserInfo(name: String)
The generated code for this example won’t compile because it misses:
JsonWriter.value(UserInfo)
JsonReader.nextUserInfo()
JsonWriter.value(ZonedDateTime)
JsonReader.nextZonedDateTime()
To make it work add @GsonSerializer
to UserInfo
and add the following code for ZonedDateTime
fun JsonWriter.value(obj: ZonedDateTime) {
value(obj.toString())
}fun JsonReader.nextZonedDateTime(): ZonedDateTime {
return ZonedDateTime.parse(nextString())
}
Abstract Types
Reading and writing abstract types is necessary and possible, consider this:
@GsonSerializer
class AppState(val requests: Set<NetworkRequest>)@GsonSerializer
interface NetworkRequest@GsonSerializer
data class LoginReq(val username: String, val password: String) : NetworkRequest
The processor checks the type-hierarchy and writes the object and type-name into an array. I chose an array over an object so I wouldn’t have to cache anything in case the type-name arrives after the data during deserialisation.
fun JsonWriter.abstractValue(obj: NetworkRequest?) {
if (obj == null) {
nullValue()
return
}
beginArray()
when (obj) {
is LoginReq -> value("LoginReq").value(obj)
else -> throw IllegalStateException("type $obj not annotated")
}
endArray()
}/* deserialize */
fun JsonReader.nextNetworkRequest(): NetworkRequest {
beginArray()
val typeName = nextString()
val obj: NetworkRequest = when (typeName) {
"LoginReq" -> nextLoginReq()
else -> throw IllegalStateException("unknown type: $typeName")
}
endArray() return obj
}
Graphs and circular objects
Using another annotation @ParentRef
, the processor is able to process circular graphs, like a leaf-node having a references to it’s root
@GsonSerializer
class Root(var leaf: Leaf?)@GsonSerializer
data class Leaf(@ParentRef val root: Root, val number: Int)
The deserialiser for Leaf
returns a function now:
fun JsonReader.nextLeaf(): (root: Root) -> Leaf {
var number: Int? = null beginObject()
while (hasNext()) {
when (nextName().toLowerCase()) {
"number" -> number = nextInt()
else -> skipValue()
}
}
endObject() return { root->
Leaf(
root = root,
number = checkNotNull(number) {
"error parsing Leaf, field not found: number: Int"
}
)
}
}
For type-safe deserialisation to work, the field in the parent-object must be either nullable or a collection-type:
fun JsonReader.nextRoot(): Root { var leaf: ((Root) -> Leaf)? = null
beginObject()
while (hasNext()) {
when (nextName().toLowerCase()) {
"leaf" -> {
leaf = nextOrNull(JsonReader::nextLeaf)
}
else -> skipValue()
}
}
endObject() /* construct parent */
val obj = Root(
leaf = null
) /* set child */
obj.leaf = leaf?.invoke(obj) return obj
}
Summary
This was a short overview about my new method of serialisation. I can’t share any code yet but the principles are clear.
The choice of compile-time-generation and Kotlin with it’s extensions methods and lambdas makes it possible to easily create convenient and fast serialisation-code.
Assuming all necessary data to be passed in the constructor helps keeping everything safe and enforces separation of initial from generated data.
I hope you can take some ideas with you. Happy coding.