Death to JSON

For the love of robust mobile apps, let’s stop using JSON as serialisation protocol.

My beef with JSON is this. It is too dynamic. For each key in a dictionary, you may get a string, a number, a boolean, or another object. This makes it very difficult to reason about the behaviour of software. A method that receives a JSON object, would have to guard against unexpected types.

As you know, Swift is the next programming language for iOS. If you are an iOS developer, you would have noticed how grossly incompatible JSON is with the language. It is impossible to serialise a JSON object into a Swift Dictionary safely. NSDictionary does not enforce types, which defeats the purpose of using a strongly typed language. It is anti-pattern to use NSDictionary with Swift.

Objective-C on the other hand is more compatible with JSON, but it has the same problem with types not being enforced. We can have an NSString pointer pointing to a NSNumber, and all would be fine until we call a NSString method on it.

This is the same for Java. We would most likely be deserialising a JSON object into a JSONObject, which is equivalent to Map<String, Object>, and that is problematic because it is not clear what Object is. Even if we knew the type of a value of a key, human error could change {“price”:“3.99”} to {“price”:3.99}.

On native mobile apps, we are working with languages that can enforce types for safety, and yet we are unable to take full advantage of them. What a shame.

Structured Binary serialisation

There is a better way. We can use structured(with a schema) binary serialisation protocols instead. As part of the revamp of a mobile API at work, I’ve been able to try out apache thrift.

This is how it works:

First, you define the schema in .thrift file. It looks something like this:

struct Person {
 1: string first_name
 2: string last_name
 3: i32 age
}

You then generate the class definitions in your preferred language, and import them into your project. These classes will contain the code to serialise objects and deserialise data. So for Objective-C it looks like the following:

@interface Person : NSObject <TBase, NSCoding> {
 NSString * __first_name;
 NSString * __last_name;
 int32_t __age;
.
.
.
}
@property (nonatomic, retain, getter=first_name, setter=setFirst_name:) NSString * first_name;
@property (nonatomic, retain, getter=last_name, setter=setLast_name:) NSString * last_name;
@property (nonatomic, getter=age, setter=setAge:) int32_t age;
.
.
.
@end

Notice that once deserialisation is successful, we know for sure that Person.age will return an integer, barring any assignment errors introduced by the client side programmer. In a strongly typed language like swift, it would be impossible for Person.age to return anything other than an Int.

On the API end, using Go as an example.

type Person struct {
 FirstName string `thrift:”first_name,1"`
 LastName string `thrift:”last_name,2"`
 Age int32 `thrift:”age,3"`
}

We can only assign an integer to age. In a more dynamic language like ruby, if a string was assigned to age by mistake and serialisation was attempted, it would fail, and the error can be detected earlier.

Oh yes. Did you notice? Since the serialisation and deserialisation code is generated by the thrift generator, there is no need to define key names. In other words, no more magic strings.

Usage recommendations

It will be tempting to use the generated thrift classes directly with your whole code base because it’s convenient. However, it’s best to create your own model layer and map the thrift model layer to it. The rest of the code base should not touch the thrift model layer. This helps the change set required to remain small whenever there are changes to the thrift files.

Thrift post mortem

The main motivation for trying out thrift was to reduce bandwidth consumption. We did achieve this objective, but I personally find the increase in robustness of our code base to be a bigger win.

Not having to think about edge cases where types are wrong or typos in key names allowed us to allocate cognitive resources to other areas.

The thrift files also served to be a great explicit contract between the mobile developers and API developers. We can make pull requests to change the thrift files and have discussions around it. It makes great documentation.

There is a problem with thrift we wish we had found earlier. We use thrift for a few different languages. Some languages receive more attention than others and hence are more up to date. The data type definitions are not congruent across all languages: there are 18 constants defined for types in Go for example, but only 15 for cocoa, and 16 for Java.

I’d definitely go for binary serialisation the next time I have to work with a mobile API. But I might try out something other than thrift.

More ranting

Plists are incompatible with swift too. :(