Why I wrote my own serialization package for Dart/Flutter.
When you have written an App in Flutter connecting to some sort of API, chances are you have come across json serialization at least once. If not, json serialization is about turning your data models into valid json and back, to send or receive data from a server, like in the following example:
class Person {
String name;
int age;
Person(this.name, this.age);
}// turn this
var person = Person("Thomas", 23);
// into this
var json = '{"name": "Thomas", "age": 23}';
In my first Apps, I’ve actually done this manually… and it was a tedious process. For every class you have to define fromMap and toMap methods, creating or parsing raw Maps, which can then be converted to a json string using some inbuilt functions from the Dart SDK ( jsonDecode and jsonEncode ).
class Person {
... factory Person.fromMap(Map<String, dynamic> map) {
return Person(map["name"], map["age"]);
} Map<String, dynamic> toMap() {
return {"name": this.name, "age": this.age};
}
}
This bloats up every single data class, especially with classes that have 10 or more properties. You will find that most of the time these utility functions take up more lines of code than the actual classes itself. Additionally you will have to put even more work in if you want the deserialization to be type-safe and handle invalid json objects with meaningful error messages.
Moreover, you will have to adjust your mapping functions every time you change any of the classes properties.
I often prefer coding stuff manually over using packages for everything, but in this case it just adds unnecessary and repetitive work, pollutes your codebase and increases maintenance cost.
So let’s look at a smarter way:
The general idea is straight forward, and you probably already thought about this: Just let some algorithm figure out how to do serialization based on the classes properties. In Flutter this is done via code generation aka meta-programming, where a script analyzes your code and generates other code to be used alongside it.
For Flutter, or more general Dart, there are of course existing packages out there, that do this for json serialization. The most popular ones are
- json_serializable, 1160+ likes on pub.dev
- dart_json_mapper, 130+ likes on pub.dev
Along some more smaller packages, those two share a large following on users, and the first one is even a Flutter Favorite made by the Google team itself. So, when looking at the numbers, some pretty strong options to choose from, right?
I won’t argue that these packages are bad per se, they actually work great in most use-cases and do what they are supposed to do. But, when I was working with both of these, there always were some smaller or larger things I didn’t like or some use-cases that weren’t supported — at least without some heavy workarounds.
Enough overall, that I felt the urge to do it all again. So here it is, my first dart package ever:
I tried to put the best parts from both packages together, remove the bits I didn’t like and add a bit of my own magic to create this package.
So let’s talk about what I think those packages did not so great, and how I tackled a few of the core issues.
Ugly code
I put this right as a first point, because some might not think of code style as an important factor. Who cares if code looks nice, right? Wrong. I would guess the people saying such a thing have never worked on a project with a large codebase, or over a large timeframe, or in a medium to large team. Maybe something a Product Owner would say that never wrote a single line of code themselves.
Nice code does make a difference. And ugly boilerplate code sucks, as well as language hacks or any weird syntax. I don’t want to compromise my codebase to use a serialization package, nor do I want to be forced to write my classes in a weird way.
For json_serializable, this means no boilerplate code and especially no dollar signs or private mixins ( _$MyMixin 🤢).
In contrast, I like how dart_json_mapper uses a few simple calls for encoding and decoding without any boilerplate code. Therefore, you will notice that the public facing API of my package is quite similar to the one from dart_json_mapper.
// deserialize json to type Person
Person person = Mapper.fromJson(json);// serialize a person to json
String json = Mapper.toJson(person);
Ugly code
Again? Well yes, but different. For me the generated parts of the code are still part of the codebase. It is important to me that I can read, and understand the code that is generated, as I value any other package with a good code quality. I won’t take ugly or minified generated code, where I have no clue what’s going on. I would have to trust that there is an underlying magic at play, that just does what it should.
Instead, I like to inspect the generated code and make out how it functions.
For this, json_serializable is the clear winner, since it generates readable functions for every class. dart_json_mapper on the other hand depends on reflectable and uses reflection, to analyze your classes and parse or build json. Since this is done at runtime, you have no way of inspecting any generated serialization functions, since there aren’t any.
One other downside of json_serializable is that it generates one file for each of your data class files. This will inevitably cluster your project with a bunch of files. One generated file for all models is all you need. While this one file may become quite big, it is still quickly searchable and won’t pollute your file inspector.
Compile-Time Checks
I very much like strongly typed languages, because you move part of the possible errors in your code to compile time. Any bug that is catched already at compile time is great and obviously way better then runtime exceptions.
Code-generation has the great possiblity to write extensive, but very strongly typed code, since you don’t have to care about writing concise code. As said, dart_json_mapper uses reflection at runtime, and thereby completely misses this opportunity.
Implementation Overhead
With reflectable, you are actually limited to one use per project, even when this is in a package. That’s because reflectable initializes statically, and therefore any second initialization overwrites the first one.
Also reflectable is a quite large library. Where code-generation already gives you all the analysis capabilities and meta-information on your classes that you need, dart_json_mapper completely ignores them and shifts the problem to reflectable.
What’s new
There are a few features that are unique to dart_mappable. One of them is the flexibility and extensibility built into the serialization process.
There can be a lot of custom use-cases that you might have with your models. Most of the time you will be fine with the 1-to-1 mapping of json keys to class properties, but for the few times you don’t, your serialization package should give you the possibility to change things up. This is why dart_mappable comes with custom hooks, which you can use to control and modify every part of the (de-)serialization process.
Another hot topic is generics. This is a tough one, really. When working with custom generic types, deserialization poses a huge challenge. That’s because it is actually impossible to drill down on a generic type information from a composed type. More specifically, when you have a type A<B> , there is no way to get to type B . So when you have a nested json object, you know what to decode the outer object to, but you cannot know what to decode the inner object to when using a generic decoder function like fromJson<A<B>>(jsonString) . There are event a couple of questions on Stackoverflow related to this issue, and generally the answer is “You can’t”.
Therefore, each package has its own tricks or workarounds to deal with generics. dart_json_mapper uses type_adapters, which for the most part you have to define manually for each composed type — and therefore isn’t really “generic” anymore. json_serializable doesn’t cover generics in it’s documentation, but there are a few issues on Github and Stackoverflow, as well as this article. Overall it is possible in some simple cases, but again you have to manually handle the inner type. Not exactly what I would describe as “It’s that simple”.
Unlike the others, dart_mappable lets you use generic classes naturally out of the box, without any additional work needed. Yes, you can just to A<B> object = Mapper.fromJson(jsonString) and it will work just fine. I know I just said it was impossible and it is from a language point of view, but with the power of code generation and a few tricks, I could get it to work (which I’m quite proud of 💪). I might write a separate article with a more in-depth explanation of how this works if people are interested.
Wrap up
There are some more features that I didn’t cover in this article. I documented all of them in the packages README, so if you are looking for a more technical explanation and usage guide, head over to the packages pub.dev page.
Overall, this package is still in development and there might still be some issues depending on your use-case. However, I’m using it in a few of my own projects as well as in the company where I currently work and so far it performs quite well.
If you have any further questions or feedback to the package, feel free to add an issue on Github. Any feedback is welcome 😃.
