#PerfMatters: Our Journey to Performance — WELT
„Good performance isn’t an end goal, it’s a continual journey.“ As mentioned in Dan Chilton’s post about performance improvements his colleagues at Vox Media have implemented after officially declaring a performance bankruptcy earlier this year. We, the developers at Welt, would also like to tell you about our journey to performance of our online products. Especially we would like to show you how we monitor, visualize and ensure that our products are developed with with high performance in mind.
Seeing the impact of developing websites without having any sort of performance testing and monitoring routine in a web development process was a wake up call for our team. That’s why we spent some time at the beginning of this year into research of how we can ensure that websites which we develop, fulfill the demands of our users for having a webpage that has great design and great performance at the same time.
The first step of our journey begins with getting to know the key metrics that are used to measure how fast a website performs. But how exactly can you measure performance of your website? Do you measure how fast your server processes the requests, or do your measure how fast it takes the loading bar to disappear? In the end, your web page is as fast as it’s users perceive it to be.
This is exactly what Speed Index does: it tells us “the average time at which visible parts of the page are displayed”. Meaning the time needed to load and display all the content the user is to see on your website before scrolling — the full first viewport.
Google’s recommendation (backed by several studies) for best user experience is to load and display a website’s above-the-fold-content in under one second. Considering this recommendation, many other web companies should also go ahead and declare a performance bankruptcy, just like Vox Media did. By looking at the average Speed Index of Alexa TOP 10.000 below you can see why:
Looking at the next chart below one can easily grasp the reason why the average Speed Index is so high and continues to rise:
After evaluating several tools, open-source and commercial, we agreed to go completely open source by using swiss knife of performance testing: sitespeed.io. The sitepeed.io stack includes YSlow (with an extended ruleset) and can trigger Google PageSpeed Insights (if you don’t use basic authentication) for gathering the rule-based metrics. WebPageTest is used for measuring the timings and capturing the Speed Index. After sitespeed.io is done collecting the metrics, they can be send to Graphite to be stored and aggregated and afterwards displayed using Grafana.
Since we are building our frontend using grunt.js, we used grunt-sitespeedio, in order to have a flawless integration into the CI of our fronted pipeline. Our first performance testing and monitoring workflow looked as follows:
After each build the generated sitespeed.io report is attached to every build in our CI (Jenkins) inside the build description. This way our developers get an nice overview of all the performance issues encountered in every build of our CI:
After integrating the automated workflow into our CI pipeline and having it running for several month, we realized that we were missing a couple important features.
Features like responsive design dashboards, to see how our sites perform across different target platforms. Also, we wanted our performance dashboards to show us where we stand against our competitors. At that point we decided to integrate Speedcurve into our performance testing & monitoring workflow.
Speedcurve is basically a commercial version of WebPageTest with out-of-the box dashboards which represent the performance metrics in more compelling way than it is possible with the open-source solution Grafana. Therefore we replaced our own WebpageTest & Grafana with Speedcurve. With 1 cent per check price policy of Speedcurve we are not paying more than we had to pay for having our own WebPageTest and Grafana instances hosted on AWS.
To integrate Speedcurve in our workflow we wrote our own grunt-module — Quicksilver, which wraps grunt-sitespeedio and the speedcurve-api. With grunt-sitespeedio we are enforcing our created performance budgets in the CI and locally on the machine of each developer. Speedcurve does the rest: monitoring performance regressions and notifying about major changes in the timings. Quicksilver was build to be easy extendable in case there is some new tool we would like to integrate into our performance testing chain.
One of the most important things we’ve learned is that developing for performance means that the complete team, not only the developers, must be involved. In our agile development process we introduced a frontend Performance review after each sprint. The lookback helps us keeping performance as an essential feature that we develop for our products. During the performance reviews we are discussing performance issues, and identified red flags during the past sprint.
We are also using the exceptionally handsome dashboard framework Dashing to visualize the most important metrics from the developing process of our team. With Speed Index being a very important metric, we have created a widget which displays the latest value of a site we are currently developing. Having a goal of getting the widget to change the color from yellow to green is a daily motivational boost for our developers and a reminder that performance optimization is an ongoing challenge.
Having several transparency methods set in place, I personally see very often whenever there is a discussion about implementation of a new feature, most of the times the performance impacts are taken into consideration. Having the importance of the frontend performance in the mindset of each team member results in situations where designers are asking developers (and not the other way around) about a possible performance impact.
This article was originally published on the welt.de developer blog on November 16, 2015.
