If you’re new to this whole ‘web performance’ craze, you may have noticed that it can be quite complex.
Should you measure load time? Responsiveness to user inputs? Page-to-page navigation? Do you do this for users in the Democratic Republic of the Congo, or Silicon Valley? Fiber, 4G, loon? On a Moto G4 or an iPhone 7? And which browser — one that supports
preload and HTTP/2, or something a little more retro?
Will you measure a user’s first ever visit to your site, or a return journey when their cache is nice and primed?
And will you be reporting on the average load time? (Absolutely not. The whole concept of the ‘average’ is yet another thing that Newton got wrong.) So maybe you’ll report on the median, or perhaps display a histogram of what the users actually experience?
A reasonable person might come the conclusion that this is all a bit hard, and put it on their ‘tomorrow’ list.
But it doesn’t need to be so complex, as I hope to demonstrate in this post.
Following in some footsteps
Last week, I was deep in a perfyoga session (perfyoga is like normal spandex yoga, except you replace the stretching part with the Chrome DevTools) when it dawned on me: there’s another industry that faces exactly this conundrum.
An industry with a need to boil down a complex concept such as ‘performance’ into a single metric that can be discussed and compared and tracked over time.
The automotive industry.
There are an endless number of things to take into account when measuring the ‘performance’ of a car. If you asked 10 car enthusiasts to define performance you’d probably get 10 different answers, and would be thoroughly bored by the end of the process. Just like you are with this paragraph.
You can measure top speed, acceleration, Nürburgring lap time. With cold tires or warm, half a tank of fuel or full, race driver behind the wheel or Mr Bean (I wonder… when Mr Bean crashed his McLaren, did he have to convince the insurance company he wasn’t sitting on the roof in an armchair steering via rope at the time?).
But the car industry wisely realized that if you want to have any sort of discussion and comparison of performance, you need to pick a number and stick to it. For cars, it’s the time it takes to go from 0 to 60 mph (or 100 km/h if you’re in one of the sensible countries).
So why not follow suit when trying to measure the performance of your web site? Put aside all the complexity, accept that you can’t capture everything at once, pick a number, and stick to it.
Multiple meanings of measurement
But I’m not suggesting that the only thing you ever need to measure about a web site’s performance is one single number. I’m not some sort of crazy person (despite what the orderlies allege).
Story time: I was watching a video recently, the fellow on stage was talking about ‘speed indexes’ when it struck me — like a goose to Fabio’s face — that the word ‘measure’ is used to refer to a range of different things, that serve vastly different purposes, and perhaps this is where a lot of the apparent complexity comes from.
So I think it’s worth picking apart these different meanings — like Fabio picking foie gras out of his golden locks — because this might save you from trying to ‘measure’ something in a way that it doesn’t need to be measured.
If you ask me (and you kinda did by reading this), ‘measuring performance’ can be separated into two things:
- A single measurable number that can be used to support a discussion about performance. It can be used to set priorities, and be tracked over time. It is an indicator of overall performance, and does not aim to capture all the nuance of a site’s performance. I will call this the “key performance indicator”.
- On the other hand, we need something to help us, the web developers, understand where time is being spent loading our site, in all different conditions, and to guide us when improving performance. It doesn’t need to be recorded or reported in any formal way. I will call this the “ad hoc assessment of factors impacting a site’s performance with the aim of improving said performance”. Snazzy.
This post is all about the former — hence the title promising simplicity. But with regards to the second one, I’ll give you some bullet points and a link at the end.
The key performance indicator
This is your 0–60 mph time. Some characteristics:
- It should be simple — a single number.
- It should be relevant to your site — to your users.
- It should be measured in a consistent manner.
- It should be understandable to non-developers.
Let’s say, for your KPI, you decide you will measure the time it takes until a user can begin reading the text on your page. Imagine this is 4.2 seconds for your home page.
4.2 seconds should be a number that The Head Honcho knows and cares about, that your marketing team talk about, that the SEO chap cares about. It should be written large on a whiteboard in the office. Everyone should be sad when it gets bigger, and happy when it gets smaller.
And this KPI isn’t just an excuse for a party when you break the four-second barrier, it’s also a tool to help protect the status quo.
If the advertising team want to add more ads, or the design team wants an 8K background video, you have a solid, objective number you can use to ask a meaty question such as: “are we willing to add 700ms to our load time for an extra $40k of ad revenue per month?”
You might not get the answer you want, but you’ll know that you asked the best question you could.
I truly believe that if every company did this, the world wide web would be a faster place.
I also truly believe that we should all Rollerblade everywhere; if everyone does it, no one looks like a dork.
In my imagination, I have done some great convincing and you’re sold: you will forget about trying to capture everything, you want a KPI. So…
What to measure?
Whatever makes sense to you, that’s what.
More specifically, I think you should measure the time it takes for your site to be ready for a user to do whatever most users will want to do.
I keep saying ‘your site’, but this definition will likely be different for different pages on your site. You might have a search results page, a home page, an image gallery page. Your definition of ‘ready’ will of course be different for each of these.
- If users are visiting a page to read something, it’s the point at which a user can start reading (that’s after the text has finished jumping around the page, OK?)
- If the page is a photo gallery, it’s the point at which the main photo is loaded.
- If the page is nothing more than a delivery mechanism for ads, it’s the point at which those ads are visible. #sadbuttrue
- It is absolutely not the
loadevent. It is never the
loadevent is a ridiculous thing to measure. Measuring the
loadevent is most of what’s wrong with the world today.
The next thing you’ll need to decide on is whether this number should be measured for fast machines, or for a slower CPU and network.
I’m going to politely disagree with the prevailing wisdom and say it doesn’t really matter. Do whichever makes you happiest, here are some
- Will knowing load time on a slow device change how you go about improving your site’s performance?
- If using a bigger number will help you make the case for reducing load time, use a slower CPU/network.
- If your goal is to get a crazy low number at the end of your performance tuning, then measure on a fast CPU/network.
- If you measure on a fast CPU/network, then this is probably close to what you experience on your development machine — so you can see what the final number is likely to be.
- If you want a number that more closely represents what The Masses experience, a slow network/CPU is for you (although ask yourself: why do I want this?)
None of the above are important to me, but maybe they are to you. So, keep your eyes on the prize and pick whichever will best drive performance improvements in your site.
If you’re still not sure, I suggest heads for fast and tails for slow.
Setting a performance goal
The final piece of conventional wisdom that I would like to challenge is the idea of a universal load-time goal.
Aiming to have your site load in 5 seconds over 3G on a Moto G4 seems like a good idea on the surface. But one of the websites I’m working on has a load time of (are you sitting down?)…
This is a site that gets a million hits a day. (And an example of reporting on big numbers for shock value.) A five-second load time is the pot of gold at the end of a rainbow. And we’re in an alternate universe where light doesn’t refract.
And yes, it is soul destroying. Thanks for asking.
Of course, if your site takes, say, 10 seconds to load, and you’re trying to get the budget to spend a month on performance work, then by all means share the “one-half of mobile users will give up on a site that takes more than 5 seconds to load” statistic to help make your case.
So, I think what I’m saying here is: come up with a goal that suits your site, and leave the five-second rule for floor-food.
OK that’s enough theory. How can we pull all this together into something you can go and do…
A practical approach
I’m open to other suggestions, but the lowest-effort approach I can think of is this:
performance.mark('The page is really ready')at this point.
- With this code deployed, visit www.webpagetest.org, go to the Simple Testing tab, enter your URL, and run it with the default settings.
- Go to the Details tab of the results and read the “The page is really ready” number.
- Write that number on the wall in foot-high text.
(Beware, a few people in the comments have reported that WebPageTest’s performance can be inconsistent over time.)
An aside: the site I used for this screenshot is just an experiment that I’m messing around with. It’s based on
create-react-app, doesn’t do much, and sits on a free-tier Heroku instance. So isn’t it interesting that even with a medium-weight framework like React, you can consider the baseline load time for a new website to be half a second?
In an ideal world, you would be able to account for and justify everything over an above those five hundred milliseconds.
You might as well have your KPI logging to the console so you can get immediate feedback on changes as you make them. Might as well show it as a marker on the Performance tab in Chrome DevTools too. You should probably send it to Google Analytics as a ‘user timing’ while you’re at it, so you can understand the range of real-world performance your users experience.
Measuring performance is like raising awareness for a cause — it’s pointless unless you follow it up with something.
And the only point to measuring performance is so you can follow that up and make your site faster.
Depending on where your personality lies on the fiddler-procrastinator matrix, you probably have all sorts of exciting ideas spinning around in your head about how to improve on this overly simplistic approach.
You may be thinking it’s a good idea to measure a few different metrics and give them weightings and combine them into a ‘score’. Maybe incorporate something into your build pipeline that sets off alarms if your performance goes outside some threshold. You’ll want to email weekly reports of course.
I’m not going to stand in your way, but I would suggest that you ask yourself if this is really something that needs to be done for you to make your site faster. If it isn’t then I would gently suggest, while eyeing the nearest exit, that you’re procrastinating and you should really get to work making your site faster.
Fun fact: remember that site with the 107 second load time? Well, it has an automated performance measurement and reporting system that records the load time, with screenshots, for all pages on the site, at different network speeds, with and without AdBlock, and emails the results to the whole company, every week. It’s been like this for years. And still … 107 seconds.
Raising awareness is overrated.
The ad hoc assessment of factors impacting a site’s performance with the aim of improving said performance (snazzy)
OK to recap, we want to think of ‘measuring performance’ in two distinct ways. The first is the simple KPI from the previous section.
If you’re just starting out, I don’t think that this sort of performance measurement is very important.
[ignores gasps from audience]
Because before you even start counting the milliseconds, you ought to get your house in order:
- Don’t load stuff you don’t need
- Cache everything you can cache
- Compress everything you can compress
If you’ve never done any serious performance tuning, that first bullet point alone should keep you busier than a bee in a collapsing colony (except for the queen, she’s like “fine, whatever, leave. It’ll be nice to have some peace and quiet around here”).
When your site has no unnecessary junk in its trunk, then, it’s time to learn the tools of the trade.
I hope you haven’t read all this way expecting me to explain performance measurement tools in a simple, easy-to-understand way. As the position of the scrollbar may have given away, there isn’t enough time left for that.
For you see, there’s a lot to learn, and it isn’t easy. So go and read these three pages from Google’s DevTools docs, and you will know more than 99% of people (including babies). If Firefox is more your style, the Developer Edition has some great docs as well.
And now, go forth and measure. Your goal is to get to know your site at a molecular level. When you have an intimate knowledge of what’s going on in the seconds before your site is ready, you’ll be in a position to get to work making it faster.
Thanks for reading, have a marvellous day.