Streaming Updates in JavaScript Datagrids

Niall Crosby
Mar 7, 2018 · 7 min read

ag-Grid can process over 150,000 updates per second. Together with the latest browsers and ag-Grid’s cutting-edge algorithms, let me introduce you to the fastest JavaScript datagrid in the world.

An example

Before going into all the detail, here is an example. You are invited to open the example and play around with it.

Test Results

These test were done with the following hardware / software:

The test results are as follows:

Why Streaming Large Updates is Important

Finance applications hold ticker grids as the ‘holy grail’ of grid applications. Ticker grids are the data grids that show movements in financial data. For example streaming up to date prices or market movements.

Typically the amount of data in these data grids is smaller and the amount of updates is less frequent than that in the demo here. For example if a grid has one million records then it is useless — no user is going to scroll down through one millions records. An application that presents one millions records to the end user in a flat list is a badly designed application. Likewise data updates should be infrequent enough that they can be consumed by a human. If a piece of data is changing 10 times every second, then it is useless.

So the use case presented in this demo (over 10,000 records with over 1,000 record updates per second) is either an unlikely situation or an edge case situation. So the demo can be seen as ag-Grid demonstrating it is more powerful than you will probably ever need with data updates — it can manage and update data faster than humans are able to consume.

More Than Just Updates

Look Out For More

The demo is doing more than cell updates. Here are some other things to look out for where ag-Grid is going above and beyond what other grids do.

Grouping, Pivoting or Flat

The data is initially grouped by three columns: Product, Portfolio and Book. You can change what columns are grouped and pivoted on by dragging the columns around. Alternatively you can hit the Flat, Group and Pivot buttons to set the columns to some predefined settings (the Group predefined setting is the configuration the demo initially appears in).

Grouping, Pivoting or Flat Under Load

Updates are made at the lowest level only. The values displayed at the grouped level are aggregations of the underlying values. When a value changes, the grid does ‘dirty path selection’ to work out what aggregated values need to be updated and updates just those.

One thing to try on the demo is to start the load test and then drag the columns around to play with different group and pivot combinations. Notice that this can all be done while the grid is processing the feed of data updates.

Sorting

If you click on a column the data is sorted. If you then start a load test while the grid is sorted the rows will move in real time as the data changes the sorting order. In the animated gif below the column ‘Current’ is sorted.

Sorting Under Load

The animated GIF’s frame rate doesn’t show how smooth the animation actually is, so try the sorting yourself in the demo. The grid uses the GPU (graphics hardware) for moving the rows between sorts by making use of 2D Graphics,CSS Transform and CSS Transitions. These together create very smooth animations as the rows move.

Range Selection

ag-Grid allows you to do range selections like in Excel. In the demo, try dragging the mouse over a selection of cells. On the bottom right of the grid information appears with aggregations (sum, average e.t.c.) of the values in the selected range.

Range Selection Under Load

If you select a range while the load test is running, the aggregations will update in real time with the changing data.

Why No Canvas

Let Me Explain

There are other JavaScript data grids available that claim to be fast because they are written using the HTML Canvas element. The canvas element allows for rendering using the 2D graphics library. It is a low level procedural model. If you use a grid that uses canvas then you should be aware that:

So to sum up canvas grids — they are not necessary. The example on this page demonstrates that standard HTML elements can achieve the performance required. Given ag-Grid can do the job with plain HTML then why would you want all the disadvantages of Canvas?

Grouping Updates in Messages

In a streaming environment, events are normally received across the network. This would typically be done using web sockets when the application is running in the browser.

Each event would contain one or more record updates. If many updates are streaming in, it would make sense for the source of events to batch the events up — eg rather than sending 10,000 events every second across the network with one update each, the source could send an event every 20ms which would equate to 50 messages with 200 updates each (giving still 10,000 updates per second).

The batching of events like this will not degrade the user experience as data changing every 20ms is past the speed our brains can process data, so more frequent updates would be simply lost on the human brain. However there is benefit to the computer network in batching as it will lesson congestion on the network.

In the Load Test and Stress Test each message has 100 record updates. To change this and see how it impacts performance then change the demo…

Changing the Demo

You can change the parameters of the test to see how the grid performs under what your application’s requirements are. To do this open the example in Plunker. You can edit directly in Plunker, or download the project from Plunker to your local machine and edit locally. Open up the file worker.js and edit the parameters at the top of the file.

Grid API

Users of ag-Grid might wonder if there are any tricks used to get the grid to work this fast. There are no tricks. The only item to take note of is the use of Batch Transactions with the grid API batchUpdateRowData().

Conclusion

ag-Grid can handle the performance required by modern day enterprise applications. In addition to the performance, it also includes many features that are not available in other grids.

Learn more about AG Grid — high performance JavaScript Data Grid. We write the code to visualise data in interactive tables so you can concentrate on writing the application code. We support multiple frameworks: Angular, Vue, React so you can pick the best framework for your needs.

AG Grid

We are ag-Grid and our mission is to build the best…