Using the Kotlin Serialization Library for Tough JSON Serialization
by Yarden Gavish
Serialization — we can’t pretend it’s a topic that gets many hearts racing.
However, if you’re developing an app that passes files around (e.g. through a server) or saves information to a database, serialization is unavoidable.
Here at Lightricks, we develop photo and video editing apps that do both. In our time, we’ve had first-hand experience of the best (and worst) that Kotlin serialization has to offer.
We use the Kotlin serialization library, because it’s compatible with Kotlin, has a faster runtime than Gson or Moshi, and can run on other platforms with multiplatform support. Usually, the default serializer does all of the work for us. But, when the JSON files differ from the way we want to represent them in Kotlin, there are some edge cases that force us to add complexities to the default serialization.
As part of our latest feature, we needed to create cross-platform video projects. This required creating platform agnostic JSON files representing these projects. Each platform then received JSON files, with no (or limited) ability to change them — making the conversion to in-memory video projects tricky.
In this post, we’ll explain how to handle tough serialization when dealing with JSON files that don’t naturally map directly to Kotlin objects. Specifically, we’ll explore dealing with polymorphism and changing data types during deserialization, using a JsonTransformingSerializer.
What’s up with polymorphism?
Let’s look at an example of a video project object. This is a simple video project, made up of a list of layers, specifically text, audio and video. The JSON file would look like this:
From the format of the JSON, we can see that polymorphism is needed. There must be some general abstract layer type which can be held in a single list. Each layer can be audio, video or text. So we can try and define the VideoProject as follows:
It’s made up of a list of layers, where Layer is an abstract class.
Each specific layer inherits from this abstract class. They have some of the same fields (like startTime and duration), but they have some layer-specific fields as well.
This is nice enough, but it is not (de)serializable yet.
To make this serializable, we first need to understand a little about how the Kotlin serializer handles polymorphism. The Kotlin serializer differentiates between different types by the keyword type. This indicates to the serializer which subclass to serialize. In the example above, if the type is AudioLayer, then the serializer knows to look for the volume field.
Looking at the above JSON, we see that the type field exists, meaning the Kotlin serializer will use this field to differentiate between the different subclasses. Specifically, the serializer expects the type field to exactly match a specific class.
In this case, we want to give the subclasses in Kotlin style-appropriate (camel-case) and meaningful names such as AudioLayer (rather than audio as found in the JSON). To do this, we can add the @SerialName annotation to make the layer subtypes match the type found in the JSON. For the audio layer, this would look like:
This annotation, which can be used in the class level or field level, is useful whenever we want the serialized name to be different from the Kotlin name, or to protect against any accidental refactoring. For example, if one day in the future this class is refactored to MusicLayer, the @SerialName annotation keeps the deserialization working. Without it, any name refactoring of a class or field will result in difference between the JSON and the Kotlin object, resulting in a failed deserialization.
And now back to our polymorphic serialization.
If all of the layers are known in advance, then making Layer a serializable sealed class is enough. Like this:
However, if the class cannot be sealed, we need to register subclasses manually. This would mean creating an object, such as ProjectSerializer, which includes all of the subclasses. Like this:
Then, when encoding or decoding, we would need to use the projectJson inside of ProjectSerializer, as follows:
This works because we can use the encoded field type to discriminate between the different subclasses. However, we cannot assume that this will always be the case, especially if the JSON was not created by the Kotlin serializer.
What would happen if in the example above there was no type field, but instead the field describing each layer was named layer_type? If we use the same serializer as above, we get this error:
Polymorphic serializer was not found for missing class discriminator (‘null’)
In other words, the Kotlin serializer is saying: “I don’t know which serializer to use, because I don’t have a way of telling these classes apart.”
The solution here would be to directly tell the serializer which field to use in order to differentiate between different classes. This is possible by using the annotation @JsonClassDiscriminator. In our case, it would look like this:
And the serialization works once again.
It’s worth noting that this is a problem we faced specifically because the serialization was platform agnostic. If the serialization was done by Kotlin initially, the Kotlin serializer would encode the default type field with the subclass name, and then use it to differentiate between different subtypes in polymorphism.
Transformation is Key
Let’s look at another case where a complex type written in the JSON needs to be represented differently in the object. To do this, we want to somehow transform the JsonElements before completing the deserialization. We can think of the JsonElements as the abstract building blocks of a JSON file. The transformation is like taking these building blocks and reorganizing them in the way we would like them to be.
Let’s dive in by looking at an example. Taking our VideoProject example again, let’s add a field to the TextLayer representing the center of the layer. The JSON with this new field will look like this:
We can see that the center is defined as a JSON array. From the documentation, we are told that the first value is the x-coordinate and the second is the y-coordinate.
In Kotlin, we want to represent the center field as a data class called Point, because this is the more correct abstraction from a code point of view. This would look like:
However, the Point class as defined above would be serialized like this:
And not how it is defined in the JSON (as an array).
How can we convert between these two JSON elements? The solution is to use a class called JsonTransformingSerializer. This class allows you to do any transformations needed between different JsonElements, before the serialization or deserialization. For example, it allows you to change a JsonArray to JsonObject or vice versa — which is what we need for the Point object transformation here.
Specifically, we can override the functions transformDeserialize(element: JsonElement) and transformSerialize(element: JsonElement), which do exactly what we need. Inside these functions, we need to add the logic of the transformation. In our case, this means taking the elements of the array and mapping them to the right properties in Point to create a JsonObject.
In deserialization, this would look like:
In serialization, we need to do the opposite transformation, taking the values (while omitting the keys, which are x and y) in the JsonObject and transforming them back into a JsonArray. This would look like this:
Finally, we need to use this serializer in the center field in the TextLayer.
Great! We once again have the serialization working!
This interface can be useful in many other scenarios, such as omitting or manipulating certain values, or array wrapping/unwrapping.
Conclusion
I sincerely hope that you will always be able to use the default serializer for all your serialization needs. But I know that sometimes, as in the case of cross-platform JSON files, the default serialization is not quite good enough. In those tough cases, I hope you find these JSON serialization examples helpful.
For more information, I also recommend checking out the full Kotlin serialization guide (here) which is full of lots of examples and explanations.
Happy serialization!
—
Create magic with us
We’re always on the lookout for promising new talent. If you’re excited about developing groundbreaking new tools for creators, we want to hear from you. From writing code to researching new features, you’ll be surrounded by a supportive team who lives and breathes technology.
Sound like you? Apply here.
