EXPEDIA GROUP TECHNOLOGY — SOFTWARE

Measuring Interaction Performance with FPS Data

How janky is your site?

Understanding browser interaction performance is a fundamental insight for smooth user experiences

Faster load performance leads to more customer conversion at Vrbo™ (part of Expedia Group™) and every other e-commerce site. Carlos Moro demonstrates this well in Web Performance vs. User Engagement. Collecting metrics such as DOM content load (DCL), time to interactive (TTI), and first contentful paint (FCP) is straightforward — they are simple timestamps relative to the start of loading a web page. But what about the interactive performance of your site? How sluggish do your customers perceive your site to be when they touch, scroll, click, and animate UI elements?

For the purposes of this article, let’s refer to degradation of interaction performance as “jank”. MDN has a definition handy as well:

Jank refers to sluggishness in a user interface, usually caused by executing long tasks on the main thread, blocking rendering, or expending too much processor power on background processes.

Consider the following demo of jank:

Interactive Demo of Jank: https://expediagroup.github.io/fpsmeter/

The above demo is a carousel that illustrates how jank can be measured with frames per second (FPS) data. We consider jank to be user friction and continually streamline our code to run as efficiently as possible.

We consider jank to be user friction

Here’s how we measure jank at Vrbo™ in the first five seconds of interaction:

• Use window.requestionAnimationFrame to calculate FPS (source: FPSMeter)
• Collect average FPS every 500ms starting DomContentLoaded
• If customer switches tabs, halt FPS collection

Visualization of measuring average FPS every 500ms

When we collect this data, we use Grafana to process regions and graph trendlines that we can use to see a real user metric (RUM) approximation of the jank a user experienced when loading the page.

Visualization of FPS RUM data between 3s and 3.5s

The above graph tells us over the last week, between 3.0s and 3.5s of using one of our sites, the frame rates were:

• 5% of customer page views had ~5 FPS or worse
• 10% of customer page views had ~11 FPS or worse
• 25% of customer page views had ~30 FPS or worse
• 50% of customer page views had ~55 FPS or worse

This data shows us how our customers are experiencing our site, and it is a great approximation of how responsive our site is at given time windows. 25% of our customers after three seconds are likely perceiving some jank — and possibly extreme jank.

Let’s look between 4.5s and 5s:

Visualization of FPS RUM data between 4.5s and 5.s

• 5% of customer page views had ~13 FPS or worse
• 10% of customer page views had ~29 FPS or worse
• 25% of customer page views had ~50 FPS or worse
• 50% of customer page views had ~58 FPS or worse

After 4.5s, 75% of our customers are experiencing a good interaction performance of 50 FPS or better. You must understand this data to understand how your customers are experiencing your site.

What causes jank and how do I fix it?

“The pixel pipeline” — from Rendering Performance

I’ll keep this section brief, as many others have written at great depth on this topic, but jank is generally caused by scripting, style calculations, layout, paint, or compositing. JavaScript tends to degrade render performance because its handlers can frequently and exponentially trigger the other operations.

Jankfree.org is a great resource for best practices around improving your site’s interaction performance. There are plenty of other resources available for debugging and fixing rendering performance issues that I won’t cover in this article.

How can I detect user jank?

We’ve now open-sourced our JavaScript utility that we use to make FPS RUM collection easy!

Here’s a brief snippet of how you could integrate FPSMeter to collect 10 samples of FPS data every 500ms:

import FPSMeter from '@vrbo/fpsmeter';
const meter = new FPSMeter({
    calculatePerMs: 500,
    maxCalculations: 10
});
meter.start();

function onSummarizeClientTelemetry() {
    let summary = {};
    // stop meter
    meter.stop();
    // collect fps windows into string attribute
    summary.fps = meter.fpsWindows.join(',');
    return summary;
}
// Demonstrates collection
setTimeout(() => {
    let summary = onSummarizeClientTelemetry();
    console.log(summary.fps); // prints "60,60,60,60,60,60,60,60,60,60" if perfect client performance
}, 10000);

If the user loses focus on the page, FPSMeter will halt collection to ensure the data being collected is clean.

How to determine FPS sample size

Through trial and error, we determined that measuring FPS in 500ms sample windows was an effective measurement of customer interaction performance. We tried other sample windows (such as 5s), but found that shorter windows gave much richer distribution and deterministic results.

Ultimately, if you want to measure interaction performance most effectively, you will need to tweak your sample window size and number of samples to suit your performance measuring needs. Simply adjust the calculatePerMs and maxCalculations in FPSMeter to tweak your sampling strategy. With some trial and error, you should be able to get the best quality readings.

When observing performance of your sites, interaction performance is an important aspect of the user experience. Collecting FPS RUM data is a great and inexpensive way to approximate interaction performance without adding overhead to your customer’s device. By using FPSMeter with some data processing and a graphing tool, you can create a key performance view and get a more holistic view of your site’s performance.

Credits

• Brian Quinn —Co-author of this article, involved in early research, developed initial version of FPS graph visualizations
• Carlos Moro — Involved in early research, developed current FPS graph visualizations