Nerd For Tech
Published in

Nerd For Tech

JSON Serialization in Ktor

Introduction

As part of my series “An opinionated Kotlin backend service”, I was checking out different ways to serialize/deserialize JSON payloads.

According to https://ktor.io/docs/serialization.html these are the supported converters:

  1. Gson
  2. Jackson
  3. kotlinx.serialization

I tested #1 and #3 and because I like challenges from time to time I threw Moshi into the mix. I left out Jackson because… well why not ;-).

My test was simple. I wanted to serialize/deserialize this data structure(s):

data class Customer(
var customerUUID: String,
var createdAt: Instant,
var modified: Instant,
var firstName: String,
var lastName: String,
var language: Language,
var account: Account,
)
data class Account(
var accountUUID: String,
var createdAt: Instant,
var modified: Instant,
var status: AccountStatus
)

The serialization part retrieved a list of customers (ArrayList<Customer>), each one referencing an account -> GET.

The deserialization part created an account first then a customer referencing that account -> POST.

Intermezzo

Custom converters can be registered with Ktor easily. They are just another Ktor Feature like routing, logging or error handling:

(in other frameworks a feature would be called middleware, filter or delegate handler)

A feature can be installed using the install function. For our purposes we install a ContentNegotiation feature like so:

install(ContentNegotiation) {
register(ContentType.Application.Json, CustomJsonConverter())
register(ContentType.Application.Xml, CustomXmlConverter())
}

All three tested converters have extension functions to simplify the registration:

install(ContentNegotiation) {
gson() // Gson converter
json() // kotlinx.serialization converter
moshi { } // Moshi converter
}

Of course you only install one of them at a time.

Gson

Dependencies

Add this dependency in your Gradle build file:

implementation(“io.ktor:ktor-gson:$ktor_version”)

Installation

As mentioned above this is all you need to install the GSON converter:

install(ContentNegotiation) {
gson()
}

The gson function gives you access to the GsonBuilder to customize the converter:

install(ContentNegotiation) {
gson {
setPrettyPrinting()
disableHtmlEscaping()
registerTypeAdapter(yourClass, yourCustomAdapter)
}
}

Customization

Running my tests out of the box. I got this for the customer/account data structure:

[
{
"customerUUID": "bbbe4f11-c36d-49cd-a031-787d6559d3ea",
"createdAt": {
"seconds": 1617802016,
"nanos": 609375000
},

"modified": {
"seconds": 1617802016,
"nanos": 609375000
},

"firstName": "Emanuel",
"lastName": "Moecklin",
"language": "en",
"account": {
"accountUUID": "20836570-d4ef-420e-b500-7b7f6516e1a8",
"createdAt": {
"seconds": 1617802016,
"nanos": 415274000
},
"modified": {
"seconds": 1617903421,
"nanos": 798094000
},
"status": "Created"
}
},

As you can see the Instant object was serialized but in a rather unconventional way. To remedy that I added a custom adapter:

object GsonInstantAdapter: TypeAdapter<Instant>() {
override fun write(writer: JsonWriter, value: Instant?) {
runCatching {
writer.value(value.toString())
}.onFailure {
throw IllegalArgumentException(it.message)
}
}

override fun read(reader: JsonReader): Instant? {
return runCatching {
Instant.parse(reader.nextString())
}.throwOnFailure()
}
}

Please note that I used https://github.com/michaelbull/kotlin-result to handle the results of the serialization/deserialization (runCatching, onFailure).

throwOnFailure is an extension function I use in combination with Valiktor (see my article about object validation: https://medium.com/nerd-for-tech/object-validation-in-kotlin-c7e02b5dabc). The following code leaves out the Valiktor part, the full code will be in my main article “An opinionated Kotlin backend service”.

fun <V> Result<V, Throwable>.throwOnFailure(): V? {
onFailure {
val message = it.message ?: it.javaClass.simpleName
throw IllegalArgumentException(message)
}
return component1()
}

Now we need to register the adapter with the GsonBuilder:

gson {
setPrettyPrinting()
disableHtmlEscaping()
registerTypeAdapter(Instant::class.java, GsonInstantAdapter)
}

The result:

"createdAt": "2021-04-09T19:55:13.571019Z",
"modified": "2021-04-09T20:22:58.167671Z",

kotlinx.serialization

Dependencies

Kotlin’s own serialization plugin needs to be configured like so:

plugins {
kotlin("plugin.serialization") version "1.4.32"
}

It also needs this dependency:

implementation("io.ktor:ktor-serialization:$ktor_version")

Installation

install(ContentNegotiation) {
json(Json {
prettyPrint = true
isLenient = true
})
}

Customization

Running this will throw an exception:

kotlinx.serialization.SerializationException: Serializer for class 'Customer' is not found. Mark the class as @Serializable or provide the serializer explicitly.

So we add a @Serializable annotation to each data class:

@Serializable
data class Account(
@Serializable
data class Customer(

Now we get a compile error:

Serializer has not been found for type ‘Instant?’. To use context serializer as fallback, explicitly annotate type or property with @Contextual

To remedy this I wrote a serializer for the Instant class (analogous Gson):

@OptIn(ExperimentalSerializationApi::class)
@Serializer(forClass = Instant::class)
object InstantSerializer : KSerializer<Instant> {
override fun deserialize(decoder: Decoder): Instant
= Instant.parse(decoder.decodeString())

override fun serialize(encoder: Encoder, value: Instant) {
encoder.encodeString(value.toString())
}
}

To register the serializer each property needs to be annotated like this:

@Serializable(with = InstantSerializer::class)
var createdAt: Instant?,
@Serializable(with = InstantSerializer::class)
var modifiedAt: Instant?,

Alternatively you can add this at the top of the file (more convenient if you have multiple properties):

@file:UseSerializers(InstantSerializer::class)

The second option can be used to register multiple adapters:

@file:UseSerializers(InstantSerializer::class, AnotherSerializer::class)

Moshi

Moshi isn’t officially supported by Ktor but Jackson is, so why not Jackson but Moshi? The answer is simple: I wanted a library that is modern with a concise and fluent API and supporting Kotlin explicitly. Both libraries are foremost Java libraries but Moshi certainly has the better Kotlin support.

Jackson has been known as “the Java JSON library” or “the best JSON parser for Java”. Or simply as “JSON for Java”.

(https://github.com/FasterXML/jackson)

Moshi is a great JSON library for Kotlin. It understands Kotlin’s non-nullable types and default parameter values. When you use Kotlin with Moshi you may use reflection, codegen, or both.

(https://github.com/square/moshi)

Dependencies

Setting up Moshi wasn’t as straight forward as the other two libraries. TBH it as quite an ordeal to make this work so let me walk you through the process.

First you add the standard dependencies:

implementation("com.squareup.moshi:moshi:1.12.0")
implementation("com.squareup.moshi:moshi-kotlin:1.12.0")

Then you need to install Moshi as a Ktor feature. I found this library that bridges the gap between Moshi and Ktor: https://github.com/rharter/ktor-moshi. Unfortunately after adding it I got an exception:

Using blocking primitives on this dispatcher is not allowed. Consider using async channel instead or use blocking primitives in withContext(Dispatchers.IO) instead.

After some research I fixed it by running some of the conversion code off the calling thread:

After adding this Moshi can be installed as Ktor feature:

install(ContentNegotiation) {
moshi { }
}

When running this and trying to serialize I got:

Cannot serialize Kotlin type Customer. Reflective serialization of Kotlin classes without using kotlin-reflect has undefined and unexpected behavior. Please use KotlinJsonAdapterFactory from the moshi-kotlin artifact or use code gen from the moshi-kotlin-codegen artifact.

After some digging I found that there are two ways to have Moshi create adapters automatically:

  1. Reflection
  2. Codegen

Moshi Reflection

As the name implies, it’s using reflection to generate adapters. All you need to do is add the KotlinJsonAdapterFactory:

moshi {
addLast(KotlinJsonAdapterFactory())
}

Note that we use addLast instead of add because Moshi adapters are ordered by precedence and you want your own adapters to “be first”.

Of course now we also need an adapter for the Instant class:

object MoshiInstantAdapter : JsonAdapter<Instant>() {
@FromJson
override fun fromJson(reader: JsonReader): Instant =
Instant.parse(reader.nextString())

@ToJson
override fun toJson(writer: JsonWriter, value: Instant?) {
writer.value(value.toString())
}
}

The adapter needs to be registered:

moshi {
add(MoshiInstantAdapter)
addLast(KotlinJsonAdapterFactory())
}

And we’re good to go, you’d think… but no:

No JsonAdapter for class java.util.ArrayList, you should probably use List instead of ArrayList (Moshi only supports the collection interfaces by default) or else register a custom JsonAdapter.

After a lot of digging I found that Moshi simply doesn’t support ArrayLists out of the box and I found this answer on StackOverflow https://stackoverflow.com/a/61272734/534471:

Registering the ArrayListAdapter and we’re finally and truly done:

moshi {
add(MoshiInstantAdapter)
add(MoshiArrayListJsonAdapter.FACTORY)
addLast(KotlinJsonAdapterFactory())
}

Moshi Codegen

Moshi reflection is slow and adds a good amount of extra code (kotlin-reflect library) -> important for mobile apps. Codegen on the other hand uses an annotation processor. It generates a small and fast adapter for each of your Kotlin classes at compile time.

To use Codegen we need to add

To use Codegen we need to add the Kotlin Annotation Processing Tool (kapt) and then the Codegen annotation processor:

plugins {
kotlin("jvm") version "1.4.32" apply false
kotlin("kapt") version "1.4.32" apply false
}
kapt("com.squareup.moshi:moshi-kotlin-codegen:1.12.0")

Please note that kapt needs to come after the jvm plugin. Also note that it’s kapt(“…”) not implementation(“…”). I wasted time because I used implementation (copy & paste error) and wondered why it didn’t generate the adapter classes (got “Failed to find the generated JsonAdapter class”).

We also need to annotate each data class with @JsonClass(generateAdapter = true):

@JsonClass(generateAdapter = true)
data class Account(
@JsonClass(generateAdapter = true)
data class Customer(

That’s it.

Conclusion

  • Gson (19.4k stars, 3.8k forks, last commit May 13, 2020):
    easy to setup, the only lib that worked out of the box (kind of). I definitely spent the least amount of time, getting Gson serialization up and running. If you don’t want to waste time, getting the bleeding edge tech to work and don’t care about maximum performance, Gson is the way to go.
  • kotlinx.serialization (3k stars, 320 forks, last commit April 5, 2021):
    the new sheriff in town. It took me longer to set it up than Gson but it’s obviously made for Kotlin and as Kotlin plugin does its magic at compile not runtime. It also supports Kotlin multiplatform, you can use it for Android, Ktor, Gradle/Maven, Kotlin/JS, Kotlin/Native etc..
    The only “beef” I had with kotlinx.serialization is the fact that it supports classes with @Serializable annotation but doesn’t support Serializable classes (the ones extending the Serializable interface). I understand that creating serializers at compile time is impossible for interfaces @Serializable annotation is ignored because it is impossible to serialize automatically interfaces or enums. Provide serializer manually via e.g. companion object)” but there could be a reflection based solution for interfaces at runtime.
  • Moshi (7.2k stars, 598 forks, last commit April 10, 2021):
    was hard to set up, the documentation is confusing if you’re not targeting its default platforms (Android and Java). Right now I don’t see a reason why I would take Moshi over any of the other two libraries. The project is quite active though so maybe it will evolve to offer better/easier support for Ktor and Kotlin multiplatform.

I personally will use kotlinx.serialization. I didn’t find any show stoppers so far and it seems to be a very active active project although I will definitely keep an eye on Moshi and where it’s going.

Note: while researching https://github.com/papsign/Ktor-OpenAPI-Generator (article to follow), I got the following error with kotlinx.serialization

Exception occurred: Serializing collections of different element types is not yet supported

I switched back to using Gson and everything works as expected. Using older but battle proven libraries seems to have its advantages.

Addendum: performance

Comparing the performance of serialization/deserialization for the different libraries seems is a challenging task. I noticed major differences between different benchmarks. Some see kotlinx.serialization as the winner, some Moshi and others Jackson:

If performance is important to you, you might want to run your own benchmarks for your specific environment and for your specific use cases. Surprisingly my own benchmarks put Gson ahead of Moshi (with reflection being slower than codegen) while kotlinx.serialization came in last but since I spent about 15 minutes on those benchmarks I don’t really trust them.
My experience with complex backend services is that network performance / latency is usually the most time consuming aspect (and sometimes database performance) while serialization is really negligible so I won’t have sleepless nights pondering the performance of different serialization libraries.

Feel free to provide feedback to the article and happy coding!

--

--

--

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To know more about us, visit https://www.nerdfortech.org/.

Recommended from Medium

From Flutter to Flight 6: Remodeling

Building E-commerce with Django-Graphene (GraphQL) in 185 lines of Code

Nyxo.app website is now open source

Did You Know You Could Animate an SVG Like This?

STARTING NEW LANGUAGE

Build a Distributed Code Hosting Platform that Supports Millions of Users

Red Hat OpenShift Serverless..

Code is Law is Code

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Emanuel Moecklin

Emanuel Moecklin

More from Medium

Quick Note: Custom WebView Class Won’t Display Virtual Keyboard After Kotlinized

Integrating a Version Catalog

C. Architecture Components — 1. UI Layer — 1.1 View Binding

Android — Intercept POST form data inside of WebView