Measuring JS code performance. Part II: interaction speed.

Photo by Alternate Skate on Unsplash

We need to worry about code performance — everybody knows it. But how FE developers can accurately benchmark their code to estimate how big is the performance effect of a given change to the codebase?

In the first part of the article we’ve used react-benchmark to render a given React component N times in order to measure the CPU/RAM consumption. What I love about this method is that everything is done automatically: webpack is used together with benchmark.js in order to create a benchmark bundle, and then puppeter is used to run this bundle in a real browser. This makes the benchmark really accurate, because everything is measured by the computer and there is no space for a human mistake.

On the other hand, there is also a disadvantage: only CPU/RAM consumption is measured in the synthetic benchmark. And if CPU consumption is 30% higher, what effect it will cause for an end user? Will there be any noticeable difference for somebody with a low-end device? Will interactions (e.g. button clicks) take longer time to respond?

To answer those question I propose you to run today another kind of benchmark: we will run our application in Chrome ourselves and will measure some user-action performance metric, like, for example, button press interaction reaction time.

Practical example

We will continue with the same example, as we’ve used in the first part: we have a DOM operations heavy React-based application, and we want to estimate if it is ok to integrate it with the OpenReplay system in terms of performance.

Phase II: UI interaction measurement

Ok, the previous benchmark has proved that OpenReplay does affect the system both in terms of CPU and RAM. And that number of DOM nodes is correlated with the performance penalty. We’ve noticed a 2–3x times decrease in CPU during our experiments, so we expect the user interactions to become slower also by 2–3x times.

Methodology

1. To render a new component inside a real web app. This component just contains one button.
2. When the button is clicked, the component adds 500 DOM nodes to the DOM. And yes, it is your favorite beer song 🍻
3. Chrome Performance profiling should be started in CPU throttling mode (4x)
4. The button is manually clicked four times in a row with a 5s interval
5. Performance of click event + noticeable events around is measured
6. Each test should be performed six times: three times w/out OpenReplay (control) and three times with OpenReplay (test)

The recording shows only one click, but I will extrapolate this to 5 clicks in a row to have a better accuracy

Hint!

It is better to save the profiling recordings to be able to analyze ’em later.

Test component

Starting with implementing a test component. Yes, it is the same component which renders the same children song, so this step is pretty easy: we have everything ready to use.

Running it!

Let’s run the benchmark and build a table with the results:

Click event reaction time with and w/out OR enabled

Ok, the results are gathered, so but it is better to visualize ’em. We will use only the 4th click to see the biggest difference.

Wow! The results are perfectly correlated with the benchmarking results received from react-benchmark. A 2–3x CPU performance decrease resulted in 3 times longer reaction time. It improves my confidence, that those benchmarks were measured accurately.

Unfortunately, those are bad news for me: we are not able to integrate OpenReplay for everybody. But we can enable it only for customers with high-end devices.

What have I learned today?

Today, we’ve learned that any given react component can be benchmarked:

1. Automatically, by running it inside headless Chrome with react-benchmark (part I)
2. Manually, by running it inside browser and measuring one of the UX metrics, such as button click response time

The second way takes more time and hard to be automated, but I believe it is worth to run both ways and then to merge the results. This way, you can get a full picture of your JS code performance drawbacks.

To get more accurate results, please remember these simple rules:

1. Enable CPU throttling to make test results less dependent on other processes of the OS.
2. Think about the use case you are benchmarking. If you test an error reporting system, your test component should throw a lot of Error objects. If you test an analytics system, your test component should have a lot of events triggered. If you test some data transforming function, please test it on a really huge data structure.
3. Close all the applications on your computer before starting the benchmark. Of course, you can not close everything (e.g. you can not close Finder or your firewall installed by the security department), but try to close as many programs as you can.