Save my ass: benchmark of json deserializers on Android
I think it’s safe to assume that the vast majority of apps has some kind of connectivity. Whether it’s content, tracking or remote config, most apps exchange some bits and bytes with the outer world. Once your app receives its desired data, it needs to serialize it somehow. Most REST APIs I know use Json. It’s lightweight, simple, elegant and easier to parse than XML. In cases where content is loaded exclusively remotely, parsing can become your optimization bottleneck. At stanwood, one of our white label apps is just like that. You swap the endpoints, change some colors and you’ve got yourself a custom news app.
Our default json parse is the good ol’ Gson. But since we’re all a bunch of nerds here at stanwood, we’re always on the lookout for exciting stuff. Recently, a new, experimental feature has been announced for Kotlin: serialization. I thought it’s the best occasion to reevaluate today’s Android json scene and take the most popular libraries for a spin.
The benchmark
Test candidates
Along with Kotlin’s new feature, I selected three other popular json parsing libraries used on Android:
Test data
I have downloaded three sample .json files from https://jsonplaceholder.typicode.com (thanks a lot!) that I used as test cases:
Test environment
I made a test app that used all four libraries to deserialize all three test cases. I asked my friends at stanwood to download it. To make things simple, the app automatically uploaded anonymized results to Firebase so that I could later analyze them.
- Each test case was run 640 times.
- 1920 results were gathered.
- Data from 14 unique devices was collected.
- 8 versions of Android system were used.
Evaluation
Execution time was meassured with Kotlin’s stdlib measureTimeMillis:
// Kotlin
measureTimeMillis {
JSON.parse(Foo::class.serializer().list, json)
}// Gson
measureTimeMillis {
val typeToken = object : TypeToken<List<Foo>>() {}.type
gson.fromJson<List<Foo>>(json, typeToken)
}// Moshi
measureTimeMillis {
moshi.fromJson(json)
}// Jackson
measureTimeMillis {
jackson.readValue<List<Foo>>(sample1json)
}
Results
Below is the average deserialization time for all test cases, devices and operating systems. We can immediately see a bunch of interesting observations.
Observation 1: Jackson sucks
It’s hard not to notice the big spike in the middle of the chart. While the rest of the libraries are all in the same range, Jackson was the only one that visibly stood out from the rest. Although my test data was quite simple, I initially suspected that this might have been influenced by some kind of edge case, Jackson was having a problem with.
That hypothesis proved to be false. In every test case, Jackson was consistently worse than any other tested libraries. This led me to another suspicion: What if Jackson is only slower on certain kinds of Android versions? What if it works fine on Oreo but struggles on Lollipop?
Turns out I was almost right. The gap between Jackson and the rest is huge on 4.1.2. It get’s smaller as we approach Android Pie, but it doesn’t disappear completely. It’s still very visible. Not a single combination of device, OS and test case helped Jackson.
Observation 2: Kotlinx.serialization, Gson and Moshi have all similar performance
Once we get rid of Jackson, we instantaneously arrive to another conclusion, and it’s something I didn’t expect: The rest of the libraries are almost equally fast.
The difference is quite substantial on older Androids but it quickly diminishes as soon as we get closer to Android 9. Interesting to see is that there is no clear winner, though it’s important to note that Gson struggled a bit with the nested data test case.
Observation 3: Kotlinx.serialization is the fastest! (kinda)
Overall, I am very happy with the result. Although kotlinx.serialization is still an experimental feature, it can already compete with industry hits like Gson. But even though it scored best, I’m very reluctant to call it the fastest. Keep in mind that the test sample was quite small and the difference between the results were rather small as well (not talking to you Jackson). If I’d used more devices, the results might have been different.
Conclusion
Kotlinx.serialization sure looks promising! And as it approaches the 1.0 mark, I’m sure lots of people will reconsider their default json parsing library choice. As a bonus, check out this kotlinx.serialization converter for Retrofit!
