kotlinx.serialization: (de)serializing JSON’s nullable, optional properties

Alex Vanyo
Mar 30, 2020 · 4 min read

The APIs of this world can be a real mess. For any data that has newly arrived in your nicely structured, type-safe Kotlin environment, cleaning it up a bit is probably a good idea, if not absolutely necessary for your own sanity. Even for well-behaved APIs, the format can be unwieldy or a bit annoying to work with.

October 2020 edit: Updated to match kotlinx.serialization 1.0.0 interface.

The Problem

In Kotlin, one of those slightly awkward cases is handling nullable, optional properties, which the OpenAPI spec specifically allows for.

For a concrete example of when this could be useful, consider an API that supports partial updates of objects. Using this API, a JSON object would be used to communicate a patch for some long-lived object. Any included property specifies that the corresponding value of the object should be updated, while the values for any omitted properties should remain unchanged. If any of the object’s properties are nullable, then a value of null being sent for a property is fundamentally different than a property that is missing, so these cases must be distinguished.

Individually, nullable and optional properties can be handled trivially by Kotlin’s own serialization library, kotlinx.serialization. However, handling both cases simultaneously takes a little bit more work.

The solution for a required, nullable property is as straightforward as it gets, thanks to Kotlin’s null safety:

A serializable data class with a required, nullable property

Thus, { "value": null } maps to ResponseJson(null) , and { "value": "string" } maps to ResponseJson("string").

If a property is given a default value, then kotlinx.serialization automatically considers it to be optional. Thus, if our optional property isn’t nullable we can use null to represent the case where the key and value is missing from the JSON object entirely:

A serializable data class with an optional, non-nullable property

The empty JSON object { } then deserializes to ResponseJson(null) , and
{ "value": "string" } maps to ResponseJson("string"). If encodeDefaults is false, then ResponseJson(null) will also serialize to the empty JSON object { }.

Take another look at those two response objects. Once created, they look exactly the same! If we tried to combine the approaches for an optional and nullable field, we wouldn’t be able to distinguish between the case of a null value or a missing property. To do so, we are going to have to get a bit more creative.

The Solution

There are two main goals for a solution:

  • The underlying structure of the JSON must remain exactly the same during serialization or deserialization
  • It should be generic, so that any type can be represented as an optional property

To support any nullable type as the value, we can’t use null as the token to mark a property that was missing. Instead, we can set up the following sealed class to encode all of the information we need:

The OptionalProperty sealed class

To avoid changing the JSON structure, we also need a custom, generic serializer for OptionalProperty:

The custom serializer for the OptionalProperty sealed class

This KSerializer constructs and deconstructs the Present object, serializing and deserializing the underlying value object of type T using the generic valueSerializer passed in as a constructor parameter.

It might seem strange that this serializer blows up when confronted with a NotPresent value. However, we don’t have to serialize or deserialize a value that will never be represented in JSON! NotPresent will always be used as the default value to make a property optional, and we should never attempt to serialize NotPresent since it represents a value that isn’t present in the JSON.

To ensure that the last part is true for JSON objects created by us via encodeToString, the last step in our solution requires that encodeDefaults is set to false for our Json instance¹:

Putting it all together, we can now represent our nullable, optional property in a distinguishable way²:

A serializable data class with an nullable, optional property

The empty JSON object { } will serialize to ResponseJson(OptionalProperty.NotPresent), the object { "value": null } will map to ResponseJson(OptionalProperty.Present(null)), and the object
{ "value": "string" } will map to ResponseJson(OptionalProperty.Present("string")).

Even if you only have to deal with non-nullable, optional properties in the models you work with, you might still want to consider using OptionalProperty. As noted above, the easiest way to handle nullable properties and optional properties independently result in objects that look identical when created. Thus, the only way to keep track of whether a property is nullable or optional is by documentation. Enforcing this difference through the structure of the code itself is far less error-prone than having to rely on documentation.

Alex works at Livefront, where packing and unpacking data is fun!

¹: ^ If you were depending on the previous behavior of encodeDefaults with true elsewhere, you can preserve that behavior by adding another data model that exists in parallel with the serializable model. Converting between “network” models and “datalayer” models can be a super useful pattern, to do some common logic, lenient enum parsing, or creating sealed class hierarchies that aren’t directed represented as such in the JSON before doing business logic.

²: ^ If you are using the kotlin-kapt plugin (used by data binding, Dagger, etc.), then directly using @Serializable(with = OptionalPropertySerializer::class) can currently lead to a compiler error. As a workaround, create a subclass of OptionalPropertySerializer with the generic type explicitly defined:

Livefront

Thoughts from the Livefront team

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store