Streaming Updates in JavaScript Datagrids
--
ag-Grid can process over 150,000 updates per second. Together with the latest browsers and ag-Grid’s cutting-edge algorithms, let me introduce you to the fastest JavaScript datagrid in the world.
An example
Before going into all the detail, here is an example. You are invited to open the example and play around with it.
Test Results
These test were done with the following hardware / software:
- Dell XPS 15" 9550
- Intel Core i7–6700HQ with 32GB Memory
- Windows 10 Pro 64-bit
- Chrome Browser full screen on 1920 x 1080 resolution
The test results are as follows:
- Stress Test: The test passes 1,000 messages immediately to the grid with 100 record updates in each message. This means 100,000 record updates to the grid. It took the grid 562ms to process the 100,000 updates, which means a speed of 177,935 record updates per second.
- Load Test: The test passes 10 messages every second to the grid. Each message contains 100 record updates. This means the grid is processing 1,000 updates every second. This is not intended to stress the system, rather it shows how the grid manages a relatively large amount of updates. While the load test is running the grid is still fully responsive.
Why Streaming Large Updates is Important
Finance applications hold ticker grids as the ‘holy grail’ of grid applications. Ticker grids are the data grids that show movements in financial data. For example streaming up to date prices or market movements.
Typically the amount of data in these data grids is smaller and the amount of updates is less frequent than that in the demo here. For example if a grid has one million records then it is useless — no user is going to scroll down through one millions records. An application that presents one millions records to the end user in a flat list is a badly designed application. Likewise data updates should be infrequent enough that they can be consumed by a human. If a piece of data is changing 10 times every second, then it is useless.
So the use case presented in this demo (over 10,000 records with over 1,000 record updates per second) is either an unlikely situation or an edge case situation. So the demo can be seen as ag-Grid demonstrating it is more powerful than you will probably ever need with data updates — it can manage and update data faster than humans are able to consume.
More Than Just Updates
The demo is doing more than cell updates. Here are some other things to look out for where ag-Grid is going above and beyond what other grids do.
Grouping, Pivoting or Flat
The data is initially grouped by three columns: Product, Portfolio and Book. You can change what columns are grouped and pivoted on by dragging the columns around. Alternatively you can hit the Flat, Group and Pivot buttons to set the columns to some predefined settings (the Group predefined setting is the configuration the demo initially appears in).
Updates are made at the lowest level only. The values displayed at the grouped level are aggregations of the underlying values. When a value changes, the grid does ‘dirty path selection’ to work out what aggregated values need to be updated and updates just those.
One thing to try on the demo is to start the load test and then drag the columns around to play with different group and pivot combinations. Notice that this can all be done while the grid is processing the feed of data updates.
Sorting
If you click on a column the data is sorted. If you then start a load test while the grid is sorted the rows will move in real time as the data changes the sorting order. In the animated gif below the column ‘Current’ is sorted.
The animated GIF’s frame rate doesn’t show how smooth the animation actually is, so try the sorting yourself in the demo. The grid uses the GPU (graphics hardware) for moving the rows between sorts by making use of 2D Graphics,CSS Transform and CSS Transitions. These together create very smooth animations as the rows move.
Range Selection
ag-Grid allows you to do range selections like in Excel. In the demo, try dragging the mouse over a selection of cells. On the bottom right of the grid information appears with aggregations (sum, average e.t.c.) of the values in the selected range.
If you select a range while the load test is running, the aggregations will update in real time with the changing data.
Why No Canvas
There are other JavaScript data grids available that claim to be fast because they are written using the HTML Canvas element. The canvas element allows for rendering using the 2D graphics library. It is a low level procedural model. If you use a grid that uses canvas then you should be aware that:
- Styling cannot be done inside the canvas using CSS. The look and feel will be difficult to sync with the rest of your application.
- It brings Canvas and 2D drawing API technologies into the application stack. This means you will require developers with more skills than standard JavaScript / HTML / CSS.
- Customising the grid would be difficult, it is not possible to merge HTML and Canvas together. For example if you want to customise the contents of a cell, it must be doing using 2D drawing API rather than simple HTML or an Angular or React component (all of which is possible with ag-Grid).
So to sum up canvas grids — they are not necessary. The example on this page demonstrates that standard HTML elements can achieve the performance required. Given ag-Grid can do the job with plain HTML then why would you want all the disadvantages of Canvas?
Grouping Updates in Messages
In a streaming environment, events are normally received across the network. This would typically be done using web sockets when the application is running in the browser.
Each event would contain one or more record updates. If many updates are streaming in, it would make sense for the source of events to batch the events up — eg rather than sending 10,000 events every second across the network with one update each, the source could send an event every 20ms which would equate to 50 messages with 200 updates each (giving still 10,000 updates per second).
The batching of events like this will not degrade the user experience as data changing every 20ms is past the speed our brains can process data, so more frequent updates would be simply lost on the human brain. However there is benefit to the computer network in batching as it will lesson congestion on the network.
In the Load Test and Stress Test each message has 100 record updates. To change this and see how it impacts performance then change the demo…
Changing the Demo
You can change the parameters of the test to see how the grid performs under what your application’s requirements are. To do this open the example in Plunker. You can edit directly in Plunker, or download the project from Plunker to your local machine and edit locally. Open up the file worker.js and edit the parameters at the top of the file.
Grid API
Users of ag-Grid might wonder if there are any tricks used to get the grid to work this fast. There are no tricks. The only item to take note of is the use of Batch Transactions with the grid API batchUpdateRowData().
Conclusion
ag-Grid can handle the performance required by modern day enterprise applications. In addition to the performance, it also includes many features that are not available in other grids.
Learn more about AG Grid — high performance JavaScript Data Grid. We write the code to visualise data in interactive tables so you can concentrate on writing the application code. We support multiple frameworks: Angular, Vue, React so you can pick the best framework for your needs.
