Your benefits of working with JSON based virtual DOM

Many former colleagues and friends have reached out to me and asked: “How did you get this efficient and fast when working inside the UI area?”

While I have been using Javascript for 20 years, experience is really just a small point in a very fast evolving ecosystem.

The biggest impact was switching to JSON based virtual DOM, which boosted my own productivity by at least 200%. Yes, this literally means that I can develop complex UI code 3 times faster as I could before.

Since these concepts are still mostly unknown, I am writing this article to share my knowledge with you. The goal is that you can benefit as well and get to a new level of Frontend Development.

Content

1. Introduction
2. What is JSON based virtual DOM?
3. Which properties can I use on a vdom node?
4. Which implementation are we using for this article?
5. A quick overview of the neo.mjs JS class system enhancements
6. Is our virtual DOM structure a Template?
7. Constructing our first JSON based virtual DOM
8. Basic VDOM operations
9. Changing multiple Configs
10. The removeDom vdom property
11. The flag vdom property
12. Adding Components into a Component (Highlight of this article)
13. getVdomRoot()
14. Using the VDom Engine itself is optional (Highlight of this article)
15. Using Component Trees
16. Extending Classes (Highlight of this article)
17. Final Thoughts

1. Introduction

This is going to be a pretty long article. Please be open minded and curious, since these concepts are disruptive.

Before we can dive deep into the benefits, it is important to cover the basics.

I marked the highlights of this article in bold inside the Content overview. I strongly recommend to read the article from top to bottom though.

Especially the section when it comes to extending classes is mind blowing.

Using JSON based virtual DOM is especially useful, in case you want to create complex Components or Apps:

2. What is JSON based virtual DOM?

With the term I am not referring to a JSON based String, but a nested Javascript structure of Objects and Arrays, following the JSON Syntax.

Simply put:

const jsonVdom = JSON.parse(jsonVdomString);

To give you a quick example of how this can look like:

Many junior developers stopped looking deeper into this already at this point.

1. Wait, I can achieve the same in Angular, React or Vue with a 3lines pseudo XML template.
2. This does not look nice, since it is less compact.
3. Pseudo XML templates better match the DSL (domain specific language) pattern.

Since you are an open minded, curious and passionate developer, you will keep reading to figure out, what the real benefits are.

1. Agreed, it is a bit longer. It is not a template though. Meaning: it is a persistent structure, which is meant to get changed dynamically at runtime. You can change it the same way before & after a Component gets mounted.
2. This is not a comparison about Apples & Oranges. It does not matter if it looks nice at the first glance. What counts is what you can do with it.
3. If you compare a pseudo XML template to the way HTML looks (e.g. inside the Chrome Dev Tools: Inspect Element), you are correct. However, JS provides an API to work with DOM nodes, which is way more object oriented. Unless you are deeply in love with regular expressions, you will agree that Javascript is meant for working with Objects & Arrays, rather than working with Strings.

3. Which properties can I use on a vdom node?

Inside the neo.mjs implementation, we can use:

You can find the details on how these properties get applied here:

https://github.com/neomjs/neo/blob/dev/src/vdom/Helper.mjs#L262

In short:

1. tag is the most important one. You can pick any html tags or tags of custom WebComponents.
2. A vdom engine relies on each node having an id, to figure out which nodes got moved to different spots. You can manually assign ids, otherwise each node will get an automatically generated one (neo-vnode-x).
3. html: the innerHTML of your node, in case it does not have child nodes.
4. cn: the child nodes, which have the same properties available. You can nest your vdom structure as deeply as you like to.
5. style: You can define your style attribute as a String. It is strongly recommended though to stick to an Object based syntax. E.g.:
   style: {borderColor: ‘red’, marginRight: ‘10px’, marginTop: ‘-5px’}
   You can use camelCase for style attribute names.
6. height, maxHeight, maxWidth, minHeight, minWidth, width are just convenience shortcuts, which will get put into the style object.
7. You can put in any HTML tag attributes directly into a vdom object.
   E.g.: tabIndex: -1
8. vtype: ‘text’, flag & removeDom will get covered more in detail later on.

4. Which implementation are we using for this article?

Since the neo.mjs implementation is the most advanced one, I will stick to it for this article.

neo.mjs is a disruptive next generation Javascript Frontend Framework, which enables you to build blazing fast multithreading UIs.

The project is using the MIT license, so you can use it for free:

neomjs/neo

You don’t have to use neo.mjs, in case you want to use JSON based virtual DOM. The MIT license enables you to extract & use the vdom implementation as you like to. You could also create an own implementation following the same design patterns.

In case you are thinking about using neo.mjs, you should take a quick look at the Workers Setup:

The important part for this article is that the VDom Engine lives inside a separate thread. This means that all calls to the Engine are async.

5. A quick overview of the neo.mjs JS class system enhancements

ES6 is out there for a long time, so I am assuming you are familiar with the Javascript class system at this point.

The missing part are still class properties.

To fill this gap, neo.mjs has enhanced JS classes with a Custom Config System.

In short: Each class has the static getConfig() method available.

Inside of it, you can specify your class properties.

In case a config ends with a trailing underscore, this will generate afterSetX(), beforeSetX() and beforeGetX() for you.

Class configs do get applied to the class prototype via Object.defineProperties(). Meaning: configs get applied as getters & setters.

This enables you to work with configs in a consistent way, since assigning a new value will trigger the setter.

const myInstance = Neo.create(MyClass, {
    color: blue
});
myInstance.color = 'green'; // triggers the setter

As easy as this. neo.mjs is extremely config driven. You will rarely ever call methods and stick to changing configs instead.

You can use afterSetColor() to map your config into the virtual DOM. You can add it into different spots. You can check for other configs and adjust your logic as you like to.

You can use beforeSetColor() for pre-processing logic (e.g. to check if a new value is valid).

You can use beforeGetColor() to modify the value. Example: beforeGetController() → if you get a config object, create a Controller instance, otherwise return the instance.

In case you want to dive deeper into the Config System (not needed to follow this article):

https://github.com/neomjs/neo/blob/dev/src/Neo.mjs#L48

https://github.com/neomjs/neo/blob/dev/src/core/Base.mjs

6. Is our virtual DOM structure a Template?

In short: no.

As an example of a template:

https://github.com/shlomiassaf/ngrid/blob/master/libs/ngrid/src/lib/grid/ngrid.component.html

This one is most likely following best practises for the Angular framework.

You will notice:

1. There are pseudo Tags, where you add configs / properties like HTML attributes (can be tricky to distinguish what is what).
2. There are variables inside the HTML definitions.
3. There are if statements inside the Template.
4. There are for loops inside the Template.

It is compact and fits the DSL pattern, but the more complex your components get, the more if & else switches you will need.

I have seen Templates with more than 1000 lines of code.

Templates are not extendable.

Templates need to get compiled (which is a very expensive task).

Ok, there are build time compilers like Svelte out there, but this assumes that your template includes all possible states and you can no longer easily extend or modify them at run time.

In neo.mjs, there are literally no templates at all. There is no need to compile the structures, which does create a nice performance boost.

You can think of it more like manually working with a JSX based output.

You will also notice, that there are no variables, if-statements or for loops inside the JSON based virtual DOM structures.

To really understand the benefits, it is important to read the next sections.

7. Constructing our first JSON based virtual DOM

Let us continue to work with our MyClass Component file to cover the basics.

I put this Component into a container.Viewport, to render it as an neo.mjs App. The current state looks like this:

The resulting DOM looks like this:

While this example does not look fancy yet, we can already learn a lot here.

We have a super basic vdom skeleton, containing a wrapper div node and inside another div containing the text and a margin style.

Since our div is inside an App, it will get mounted right away (this is optional).

inside the afterSetColor() method, we are accessing the inner div and assign the value of our color config to its style color attribute.

To trigger a vdom update, we are calling:

this.vdom = vdom;

Keep in mind, this is a setter. It will send a post message to the VDom Engine inside the VDom worker to figure out the deltas, in case the Component is already mounted. Since it is not mounted when the initial value is assigned, it won’t trigger the Engine.

Let us add a console log:

We reload our “App” inside the Browser:
(yes, we do not need any build processes or transpilations to do this)

You will notice, that color is there as a setter (the highlighted config with the 3 dots) and its value is stored inside _color.

We can just change the value directly inside the Console:

Our Component will adjust right away:

Inside our Console, we will see our very first delta update:

So far, we needed 2 requestAnimationFrame calls. The first one for the initial mounting of the App and now the second one to change the color style.

There are 3 different ways to assign our changes to the vdom object:

this.vdom = vdom;

Using the leading underscore won’t trigger an Engine call.

this._vdom = vdom; // silent update

Since our update is async, we might want to use a callback:

this.promiseVdomUpdate().then(() => {
    // do something after the delta update got applied to the DOM
});

8. Basic VDOM operations

Let us modify our first example a bit:

We changed our color config into itemHeight and adjusted our vdom skeleton a bit. As mentioned, JS is perfect to work with objects & arrays, so we can just iterate over our items.

We do not need to keep our vdom skeleton static. Let us dynamically add a new item:

afterSetItemHeight(value, oldValue) {
    let vdom = this.vdom;

    vdom.cn.push({style: {backgroundColor: 'blue'}});

    vdom.cn.forEach(item => {
        item.style.height = value;
    });

    this.vdom = vdom;
}

We would not do this inside this itemHeight setter method, but just to get the idea.

afterSetItemHeight(value, oldValue) {
    let vdom = this.vdom;

    vdom.cn.forEach(item => {
        item.style.height = value;
    });

    vdom.cn.push(vdom.cn.shift());

    this.vdom = vdom;
}

We removed the first item and added it to the end of the items array.

Since this happens before mounting the Component, our ids will still get created ascending (1, 2, 3) from the changed item order.

Let us change the itemHeight config inside the Console:

Now our item order is 2, 3, 1. Let us look at the delta updates:

Hint: the VDom Engine can handle any combination of changes at once.

I picked the example to add the first item at the end of our array for a purpose.

vdom.cn.push(vdom.cn.shift());

You can easily do it inside the opposite direction:

vdom.cn.unshift(vdom.cn.pop());

The use case for this is infinite scrolling. Imagine you want to create a Calendar month view and when you scroll through the weeks, you want to dynamically add new week rows at the top while removing rows at the bottom (or vice versa).

You can take a look at the implementation here:

https://github.com/neomjs/neo/blob/dev/src/calendar/view/MonthComponent.mjs#L410

How would you implement a store load logic? For this, we need to loop over the data:

Since we can load our store multiple times, we do want to reset the items array before adding the new items. Well, unless we do want to dynamically add more.

9. Changing multiple Configs

Let us change the code to implement a Button containing 2 configs:
iconCls & text.

Let us take a quick look at the result:

Inside afterSetIconCls(), we want to remove the old iconCls in case it exists. NeoArray can be a convenient Helper for this.

Before we dynamically change configs, let us recall what happens:

A config change in our Button use case will trigger the VDom Engine.

This happens asynchronously (the Engine lives inside a Worker).

When you trigger a change, we will send our new vdom object, as well as a vnode object (containing the previous state) to the vdom.Helper class. This one will convert the new vdom object into a vnode and compare the old and new vnodes to figure out the deltas.

Once the operation is done, the new vnode will get assigned to our Component. If a new call to the Engine would happen, before the new vnode got back, you could get into deep trouble.

const myButton = Neo.create(MyClass, {id: 'myButton'};
setTimeout(() => {
    myButton.iconCls = ['fa', 'fa-user'];
    myButton.text    = 'New Text
}, 1000);

Doing this inside the Console:

We are in luck, the framework detects that an Update is already running for this specific Component and delays the second update until the first update is finished.

2 updates is not what we want though.

const myButton = Neo.create(MyClass, {id: 'myButton'};
setTimeout(() => {
    myButton.set({
        iconCls: ['fa', 'fa-user'],
        text   : 'New Text'
    });
}, 1000);

This is perfect.

We do get a callback as well, as you can see with the Promise <pending> log.

myButton.set({
    iconCls: ['fa', 'fa-user'],
    text   : 'New Text'
}).then(() => {
    // do something when done
});

We can also use setSilent(), in case we do not want to trigger the VDom Engine at all.

10. The removeDom vdom property

Working with completely dynamic structures can get tricky. Let us stick to our Button example, with the change that we do not want to keep and iconCls or text span node inside the DOM, in case there is no value.

Without the removeDom config, we would need to do something like:

Not including the code here, since this is not what we should do.

In case we remove the iconCls node in case there is no iconCls, we also need to re-create it in case we change the iconCls config. For the textNode, we need to check if there is an iconNode in place or not.

This logic is not even removing and re-adding the textNode.

Now imagine a more complex Component with several configs → nodes which are optional. We could use the vdom flag property to find the right spots, but this would be a nightmare anyway.

So, how can we resolve this with the removeDom property?

All we need to do is to flag the component to get removed from the real DOM, in case there is no value.

Note that iconCls has the default value of null now.

Our Button renders without the iconCls span node.

Let us change our button inside the Console and add an iconCls plus remove the text at the same time:

We removed the textNode and re-added our iconNode.

It will even keep the same id in case you remove and re-add it multiple times.

I am super excited about this feature, since it allows us to keep the vdom structure in place.

11. The flag vdom property

So far, we accessed childNodes directly like

vdom.cn[0]

This is the fastest way and perfectly fine for trivial structures.

Now imagine some “Real Live” structures:

Well, obviously you can still access your node directly.

This is no fun though. In case your structures get dynamic, it would become close to impossible.

Let us try this again using the flag property:

We import util.Vdom. We add the flag property to our target node. No worries, this one will not get into the real DOM.

Flags do not have to be unique. While getByFlag() returns the first flag it finds, you can also use getFlags() to get all nodes containing a specific flag.

Note that we did pass the vdom top level node.

In case our tree is really big, we can pass a child node as well, to reduce the search area (performance).

vdom.Util is worth a closer look, since it does provide more methods to find specific nodes (by class, style or in general any combination of node properties.

https://github.com/neomjs/neo/blob/dev/src/util/VDom.mjs

12. Adding Components into a Component

I just got asked, if it was possible to add column based filters into a table.Container.

table.Container got a new config: showHeaderFilters_: false, which will get passed to the matching header.Toolbar instance, which will then pass it to header.Button.

This way, we can just toggle it on or off on the tableContainer itself.

To really understand how powerful JSON based virtual DOM can be, let us take a closer look into table.header.Button:

https://github.com/neomjs/neo/blob/dev/src/table/header/Button.mjs#L217

You can learn a lot from this tiny code snippet.

If we set the showHeaderFilter config to true for a headerButton instance, we will check, if this Component already has a filterField instance or not.

In case this.filterField does not exist yet, we use Neo.create() to create the instance. You can use the editorFieldConfig to change any configs of the editor you like to. This does include the module, so you can easily switch to SelectFields, CheckBoxes or whatever you like best.

After the create() call, our new instance already has a vdom property, including the initial state of all passed configs.

So all we need to do is dropping the field vdom anywhere into the header.Button vdom.

me.vdom.cn.push(me.filterField.vdom);

As easy as this.

In case we set the showHeaderFilter config to false, we just use the removeDom flag again. Meaning: the JS instance (and its state) are kept, while we just remove or re-add the real DOM.

The logic to show or hide the editor fields is also worth a look:

https://github.com/neomjs/neo/blob/dev/src/table/header/Toolbar.mjs#L55

We are doing silent updates on each header button and then trigger a vdom update for the Toolbar which contains the Buttons. The result is just 1 call to the Engine.

Inside the forEach loop, we could as well use:

item.showHeaderFilter = value;

This would trigger one update for each Button instance on its own. These updates can run in parallel though, since there is no vdom intersection for each Button.

The nice part is that you are in control when and what you like to update.

I recently added the removeDom property to card layout items as well. Meaning: all inactive (not visible) cards do get removed from the real DOM.

The result was incredible: The dom for most demo Apps got reduced by 80%+, while keeping the full functionality (including the state and even scroll states).

While you can argue, that display: none nodes do not get any Browser based layout calculations, the reduction had a huge impact on the main thread memory usage, resulting in a better performance.

13. getVdomRoot()

Our self written Button example was pretty close to the real implementation:

Now, for the table.header.Button, we would like to have this button nested into a wrapper node.

You will notice that the structure is almost the same, just wrapped into the th tag.

Now we do want our afterSetX() methods to access the “same” nodes and we do want to get all our top level configs (e.g. cls on component level) applied to the button tag again.

getVdomRoot() resolves this easily. We also need to adjust getVnodeRoot() to the same level, for consistency reasons (e.g. form fields change the vnode (previous state) in case a user types into them).

14. Using the VDom Engine itself is optional

In case you are still sceptical about the JSON based virtual DOM concepts, this part should convince you.

Imagine you have a Table like this one:

In case you want to update all cells randomly, it will result in 1800 single delta update calls.

Of course, we could group them and pass the full table view to the VDom Engine, but in case we would use a SocketConnection and real get data for 1 cell at a time, we definitely do not want to parse the entire View.

We do exactly know which cell changed and what the new content should be.

Manually updating the cells results in 20 requestAnimationFrames on my machine, with 60fps around 300ms.

You can by far not even see all changes.

https://neomjs.github.io/pages/node_modules/neo.mjs/dist/production/examples/tableStore/index.html

Hint: the demo is faster having the Console closed, since 1800 update logs are slowing it down.

https://github.com/neomjs/neo/blob/dev/src/table/View.mjs#L217

In case you send a vdom object to the VDom Engine, this will create deltas and pass them to the main thread.

Obviously you can just manually create these update calls as well and send them from your scope (the App worker) directly to the main thread.

We also adjust the vdom silently, to keep the state in place.
(To do it 100% correctly, we would need to adjust the vnode as well, feature request.)

The Helix Performance demo is using the same strategy for manual updates:

In case I scroll very fast horizontally (Magic Mouse or TrackPad), I get up to 30.000 delta updates per second on my machine.

Try it:

https://neomjs.github.io/pages/node_modules/neo.mjs/dist/production/examples/component/coronaHelix/index.html

15. Using Component Trees

Component Trees are basically an abstraction layer on top of the vdom Tree.

https://github.com/neomjs/neo/blob/dev/apps/covid/view/MainContainer.mjs#L39

In case you are working with Containers, you can just drop Components (or component config objects) into the items array and use layouts (like flexbox or card).

This is really optional though. If you prefer to stick to the vdom tree itself, this is completely fine.

Since the Component Tree itself is JSON based as well, you can very easily mix it with JSON based virtual DOM.

17. Extending Classes

Of course you can extend classes in Angular, React or Vue as well, but there is simply no way to really extend your template(s).

Now, in neo.mjs, you can do this. I would not recommend to override the entire vdom object, but you can add new configs to your extended class. These can modify your vdom object to add more features (child nodes) or modify existing ones.

Of course, you can override the afterSetX() methods as well.

Well, this would most likely result in 2 calls to the VDom Engine, assuming that your parent method does change the vdom as well.

You can however use the silentVdomUpdate flag to prevent this and you will end up with just 1 VDom Engine call.

Amazing, right?

16. Final Thoughts

If you read up to this point, I am extremely proud of you.

This article was not only long, but not easy to understand at a first glance.

Even though I created this virtual DOM implementation on my own, I am still just scratching the surface of what is possible with it.

In case you do understand the basic concepts, the next step for you is to dive into the neo.mjs code base:

neomjs/neo

You are ready to take a look at some of the more advanced components now and actually understand what is happening.

You are also ready now to create your first JSON based virtual DOM components on your own!

I am looking forward to see what talented developers like you can do with it. You are very welcome to add PRs to the repo, in case you created a new example or component which you would like to show off (or just getting reviewed).

Thanks for reading this & happy coding,
Tobias