Serialising objects in JavaScript with tanagra.js

Darmok and Jalad at Tanagra.

There are many occasions on which you might want to serialise object data, then retrieve it later as an instance of the same class. Consider the following use-case: your website authenticates users when they log on, looks up the user record from the database, then embellishes it with data from a number of external services via API calls. In order to avoid having to do these (expensive) look-ups every time a user logs in, the site caches this user data; so that the cache won’t be invalidated on service restart (or perhaps to share the cache between different instances in a high-availability cluster), it is stored in Redis, necessitating that the user objects be serialised into a format that Redis can handle. When retrieving users from the Redis cache, they should be returned as instances of the user class.

This is relatively straightforward to achieve in statically-typed languages such as c#. However, the dynamic nature of JavaScript makes the problem a little trickier. Whilst ES6 introduced classes, the fields on these classes (and their types) aren’t defined until they are initialised (which may not be when the class is instantiated), and the return-types of fields and functions aren’t defined at all in the schema. What’s more, the structure of the class can easily be changed at runtime - fields can be added or removed, types can be changed, etc. (Whilst this is possible using reflection in c#, reflection represents the ‘dark arts’ of that language, and thus developers expect it to break functionality.)

tanagra.js (Trekkies will get the reference) is a new serialisation library which aims to solve some of these problems, allowing objects to be serialised, and later deserialised into instances of their original classes. It works by exposing a function to be used when defining classes, which allows the developer to specify all the types referenced by the class. (The function can be seen as equivalent to the Serializable attribute in c#.) Consider the following example:

const serializable = require('tanagra-core').serializable

class Foo {
  constructor(bar, baz1, baz2, fooBar1, fooBar2) {
    this.someNumber = 123
    this.someString = 'hello, world!'
    this.bar = bar // a complex object with a prototype
    this.bazArray = [baz1, baz2]
    this.fooBarMap = new Map([
      ['a', fooBar1],
      ['b', fooBar2]
    ])
  }
}

// Mark class `Foo` as serializable and containing sub-types
// Bar, Baz and FooBar
module.exports = serializable(Foo, [Bar, Baz, FooBar])

The class Foo references types Bar, Baz and FooBar. The call to serializable adds some static metadata to the class which stores this information, along with relevant schema metadata. (Note that this doesn’t need to be specified recursively - only the types directly referenced at this level are required.) When an instance of the class is serialised, this metadata is also serialised. Upon deserialisation, the metadata is read and used to construct an instance of the correct class.

Two methods of serialisation are currently supported: JSON and Google Protobuffers. (The protobufs serialiser is experimental, and therefore not recommended for production use.) The serialisers work completely differently: the JSON serialiser uses the built-in JSON.serialize function to store the object, whereas the protobuf serialiser generates a protobuf message on the fly, which can itself be cached in Redis or a similar datastore. Both serialisers use the the list of referenced classes appended to the class by the serializable function to look up the prototype of the class.

Serialising the data and then retrieving it again is easy:

const json = require('tanagra-json')

const instance = new Foo(bar, baz1, baz2, fooBar1, fooBar2)
const serialized = json.encodeEntity(instance)
const deserialized = json.decodeEntity(serialized, Foo)

The JSON serialiser is already usable (with a few limitations); the protobuf serialiser is experimental, with a few bugs related to deeply-nested objects.

Due to the dynamic nature of JavaScript, the serialiser needs to make some assumptions. It assumes that the structure of classes won’t change significantly at runtime. Specifically, initialising new fields with previously unreferenced classes at runtime will cause problems, as the new fields won’t be part of the class metadata used to look up the prototypes on deserialisation. (The list can of course be updated manually.)

The library does, however, support basic class versioning, so if you refactor a class after serialising an instance, then try to deserialise instance data from a previous version, the data should deserialise correctly.

I plan to extend the library to better support pre-ES6 data structures (functions-as-classes), as well as ESNext decorators and, eventually, typescript. The library is intended for use with node.js, but I plan to extend it to support client-side JS (it may be useful for caching data using cookies or local storage).

I know of no other JavaScript library that supports serialising complex, nested object data and deserialising to the correct classes. If you’re implementing functionality that would benefit from the library, please give it a try, get in touch with your feedback and consider contributing. You can reach me at luke.d.a.wilson@gmail.com.

Project homepage: http://tanagrajs.net

GitHub repository: https://github.com/lukedawilson/tanagra