Introduction to NestedTypes Data Framework

When we started our new web client project at Volicon two and a half years ago, there was no common agreement in an industry on how single-page applications should be written. Nor there were frameworks and technologies available which cover important aspects of large application design, having a reasonable learning curve. The last thing was really important; we had a team with no experience in SPA development at all, and we needed to make technology demo of the new product really fast.

I decided to start with raw Backbone, as I was pretty confident in my capability to quickly instruct team what should they do. The source of my confidence was the fact that it mimics the similar patterns which PHP/ASP web developer is used to, so it should be easy for the team to start.

You might wonder, why we just didn’t use raw JSON or JS classes. Because of two very important features which Backbone programming model promises:

1. Serializable state. JSON and objects in your data layer are not the same as JSON lacks support for many data types, such as Date. Nor can raw JSON distinguish the reference to an object with an aggregated object — they must be serialized differently. Also, your data objects might have some members which are needed on the client, but shouldn’t go to the server. Even for the simple case, it’s often beneficial to differentiate between data layer objects and JSON used to communicate with a server. Which leads us to the idea of serializable classes, which can be constructed from JSON and serialized to JSON. That’s what models and collections are.
2. Observable state. Whenever you change something in the model or collection, listeners will be notified of the change and they can, for instance, update UI. It leads to very simple and powerful UI programming pattern — instead of trying to synchronize your app state with UI via DOM manipulations, you just listen for the changes and render everything again (which is now strongly associated with React, but this pattern was originally introduced by Backbone). And it’s dramatically easier because state management itself is not hard — developers doing it for decades. But state synchronization really is.

To state it briefly, there are two major things which make web client development complex — serialization and state synchronization. Backbone attempts to address both of them (which is good). And fails.

Our team’s start with Backbone was apparently easy and we delivered the product demo really fast, however, the team quickly gets into daily troubles with collections, models, and data layer in general. There were a lot of common mistakes people doing again and again. To name a few:

• Attempt to access model attributes directly, forgetting .set( ‘x’, 1 ) and .get( ‘x’ ).
• Forgetting to wrap model defaults to the function, which ends up with shared arrays and objects in model attributes.
• Forgetting to cast Date attributes to Date and booleans to Boolean on assignments, which breaks JSON going to the server when you save the model.
• Improper handling of nested models and collections in model attributes. With Backbone, you have the complete freedom to skew the things up in many different ways. And developer always takes such an opportunity.

You would really appreciate some help from the framework when you’ve got something more complex than primitive types in your model attributes, but there are none. You’re supposed to “do things in the right way” instead.

“Do things in the right way” is a nice euphemism for the tons of stupid “service” code which you don’t want to write. But you have to, because otherwise nothing really works. Another one is “boilerplate”. You can’t just tell that the framework forcing you to write twice more code for the task for no visible reason (spending in average twice more time and having twice more opportunities for the mistake) must be an utter crap. No. You’re saying — “it has large boilerplate”.

As our team has a lot more of important things to do than “doing things in the right way”, I decided to develop the data framework which would be “boilerplate-free” and in which an intuitive “wrong way” is actually the right one. So it will take no efforts to do the things right. This framework is called NestedTypes.

Let me do the brief demonstration.

An example where everything is wrong

Here’s some Backbone code, which would be written by an untrained person. It doesn’t work. At all.

Let’s see what it will take for Backbone pro to fix it. Really. You need to see it. BlogPost definition needs to be changed so it will serialize properly. I’m using ES6 to save some typing:

And also, we cannot access attributes directly. So, “the right way”! No surprise people keep forgetting it — once typed you just wish your eyes would unsee this.

Here’s one problem, though. Changes made inside of post.author won’t cause change events for the BlogPost — no ‘change:author’ nor ‘change’ will be emitted. We need to handle it too. And it’s totally not easy to handle, it’s in fact much more tricky than override parse() and toJSON().

But I think we have enough of Backbone. So, what is about NestedTypes?

What I will tell now might be really shocking. Because in order to fix an original example, you need to delete some code. Not to add. And everything else (including direct attribute access and deep changes observation) will work exactly in the way as untrained person expects. That’s how BlogPost definition shall be changed:

How the magic works

As you probably suspect, it looks like a Backbone model, but internally it’s something completely different. The major difference is hidden behind an answer to rather a naive question — what is the model?

In Backbone, a model is the hash of the key-value pairs. You can add and remove them dynamically, and ‘defaults’ (whenever it is wrapped in a function or not) just holds its initial values set.

In NestedTypes, the model is not a hash, it’s more like a record or class. And ‘defaults’ really holds an attribute type spec; every attribute of the model you going to use must be declared in defaults.

Whenever you put the value in this declaration, it treats it as the default value. But when you put some function (as we did with Date and User) — NestedTypes assumes that it’s constructor function designating attribute’s type. And it is smart enough to serialize it properly, observe nested changes, and convert all values which you’re trying to assign to the declared type, no matter where the change comes from.

Most interesting thing is that even internally NestedTypes model is not implemented as a hash. Which leads to the fact, that it can handle updates 10 times faster than Backbone.

At this point, you might notice that NestedTypes models resemble your classes from statically typed languages — like C++ or Java. That’s right, it is exactly what NestedTypes really is. It’s the dynamic serializable type system for JavaScript. It describes serializable observable types, and guarantees you dynamic type-safety — model attributes will always hold the declared type, no matter what.

But what about the TypeScript?

TypeScript itself is a great technology. Starting from the version 2.0 which is now in “beta”, NestedTypes is entirely written with TypeScript. Speaking about type annotations, there’s an important difference with TypeScript we need to outline.

First — it’s what we’ve started from. Every type which is defined with NestedTypes is both deeply observable and serializable by default. And these dynamic type checks meant to aids serialization. With raw TypeScript, neither of that is true.

TypeScript check types statically, and never inserts type checks or casts in the code, as C++ does. It forces you to do it. Imagine an example when you have a boolean flag in your class, and you assign it with a result of some logical expression. TypeScript will likely reject to compile (as in JS logical expressions can return anything) and force you to manually insert conversion to boolean.

In NestedTypes it’s completely opposite, as it’s about dynamic, not static checks. It will allow you to do the assignment, but silently convert the value to the boolean. It will do it for all attribute types, in fact. Which will effectively guard your client/server protocol from dynamic JS semantic.

Important thing is that the protocol is guarded against both ends. Same thing will happen when the server send you something strange — NestedTypes dynamically check the type, convert the value to the proper type as it’s declared, and if it can’t — reject an update and put an error to console.