Build Frontend Performance Monitor Dashboard Using PageSpeed Insights
How we built our frontend performance monitor dashboard using PageSpeed, Apps Script, Google Sheet & Data Studio
In NE Digital, we are obsessed with providing the best user experience & speed is a key component of it. We are also data-driven, therefore before making a change we always measure the before and after. To enable this data-driven approach for frontend performance, we recently built our frontend performance monitoring dashboard for the Fairprice website.
This dashboard helped us to understand the current values of different key performance metrics (FCP, LCP, TBT, CLS, TTI) for our website & set a baseline for those. It also enabled us to monitor continuously & easily find performance regression issues on production and fix it.
Performance is a constant process, not a one-time checklist. It requires continuous monitoring and work. A useful workflow when investing in performance is Measure, Optimize & Monitor. Addy Osmani
Widely used synthetic testing tools
These are a few of the very popular tools among frontend developers for synthetic testing and finding performance bottlenecks that we are also using internally.
PageSpeed insights
PageSpeed insight is one of the most popular tools and it combines data from the lighthouse to give synthetic testing data and CRUX data to give real user loading experience information.
Webpagetest
Webpagetest is another very popular tool and includes lots of config options such as device type, testing location, network connection, etc. It provides some very useful insights into its waterfall to performance issues. We also use its filmstrip view to focus on LCP and Speed Index.
Lighthouse
Lighthouse is inbuilt in the Google Chrome dev tool and that makes it very easy to quickly measure any of the performance metrics before/after making any change. It is very easy to use and it detects and gives suggestions on how to improve the performance of different metrics.
Problem with lighthouse
Lighthouse score varies from system to system 😈. There are many reasons for this variability. Even the score varies between different run 😭😭. This was causing some confusion and also different scores for each run was adding difficulty in setting a baseline for different metrics and performance scores.
Our solution
To solve the problem we decided to use PageSpeed insights which is a hosted lab environment and has a public API. We decided to call PageSpeed API on a specific interval and average the metric values and scores in our dashboard.
Our Approach
- Use Google Apps Script to periodically call PageSpeed API.
- Save the PageSpeed insight data using Google Sheets.
- Create a different set of visualizations in data-studio from this google sheet data.
To utilize these google services, we used our G Suite account but these services are also available to any Gmail account and after accepting the terms conditions we can quickly start to use it. We got our PageSpeed API key from here.
Now that we are clear about the flow, let’s set up the things.
Configure google sheet & Apps script
In Google Sheets, we can find the Script Editor in the Tools tab. Here we have created a sheet name results. This same name will be used later in apps script to refer to this sheet. This sample google sheet can be used for reference.
Once we click on the Script editor, it will take us to a script editor view in a new tab. Google Apps script uses javascript so it is very easy for frontend developers to use.
Next, we are going to replace the default code with the below code and save the changes. After that, we have to replace PAGESPEED_API_KEY with our own PageSpeed API key and YOUR_WEBSITE with the website URL we want to monitor.
The above code is calling PageSpeed Insight API for the given website and collecting data for desktop and mobile strategy. After that, it is inserting the data into the results sheet that we earlier created.
Let’s try to breakdown what the code is doing
Here, the above code is connecting to a google sheet and selecting the sheet named results. This will be used in different functions to perform several write operations on the results sheet.
createHeaders function creates the columns that we want to set for our sheet. Here we are setting 9 column headers and we are inserting the names of those columns using the range.setValues method. This createHeaders function we just need to run once to set the column names for the sheet.
monitor function is the function which is calling getPageSpeedInfo function to get API information from PageSpeed then feeding the data to instertDataToSheet function to insert the data to the sheet. This is the function that we are going to configure later to execute using Apps Scripts triggers.
getPageSpeedInfo function is pretty straightforward, it calls PageSpeed API and returns the result.
As the name suggests instertDataToSheet functions insert various metrics data captured via PageSpeed API to the results sheet.
We went to https://developers.google.com/speed/docs/insights/v5/get-started to see sample API response to understand which informations are captured from lighthouse and crux data and decide which ones to capture in our google sheet for monitoring.
Running code in Apps Script
Executing code in Apps script is very simple, We can find the Run function in the Run tab. It will show us all the functions from our code that can be executed.
We are going to run createHeaders function to set headers for the results sheet. and once it completes running, we will go to the google sheet and verify that the headers got added successfully.
To pull data from PageSpeed API we are going to run the monitor function and once the function finishes executing, it will append two rows in the sheet; one for desktop, another for mobile.
Setting up a trigger in Apps Script
Now that we have verified our code is working fine and data is getting saved properly in google sheets; we can set up a trigger to run this automatically like a cronjob on certain intervals. This will ensure that our website’s PageSpeed insight data is collected automatically at regular intervals.
Below you can see how to set up an hourly trigger for monitor function that we have written earlier.
Creating visualization in data-studio
Once we create a blank report in data-studio it will display us a view similar to below with many possible connectors. Since we have stored our data on google sheet, we will choose the Google Sheets connector.
Once Google Sheets connector is selected it will list the pagespeed sheet in all items. Once we click on add, it will get the data source connection.
Voila, that’s all 🥳 🥳 🚀 🚀. Now we are all set to create our own visualization from the PageSpeed API information.
Some alternatives
Here are some of the alternatives that you can also consider for your performance metric dashboard.
- Sitespeed.io is an open-source performance monitoring tool and it already includes Webpagetest, PageSpeed, Browsertime, Crux, and many other useful plugins. I 😍 sitespeed.io and some of the dashboards we use are heavily inspired by them.
- Speedcurve is a paid tool with both synthetic testing & RUM solution and very popular in the frontend community.
Future goals
- We are currently investigating how to correlate the performance metrics with the business metric to find how performance improvement translates to different business metrics such as Add to Cart, Bounce Rate, Conversion, etc.
- Another area that we are looking for is setting up alerts so we can get a notification on performance regression. Currently, the data studio doesn’t support this feature 😭😭. In the meantime, we are exploring Apps Script for setting up this feature.
