Building a Chrome Extension from Scratch

Our experience with the Chrome Extension API + tech stack decisions for building the ThousandEyes Endpoint Agent

Florent Garit, Software Engineer at ThousandEyes

When it comes to measuring how end users experience different websites, apps or services, being able to gather data from the user’s perspective is critical. One of the ways we gather that intelligence is through our Endpoint Agent, which is deployed on end-user devices. It gathers data about the browsing experience, runs network probes and Network/HTTP tests. One of the main components of the Endpoint Agent is the browser extension. We recently worked on optimizing our user session definitions, redirect flows and sessioning logic in our Chrome Extension, which led us to make some new tech stack decisions. This post will detail that process.

The Endpoint Browser Extension records navigation data on visited websites. It starts off by receiving a pre-defined list of domains to be automatically monitored, which is configured by the administrator. As soon as the user navigates to a listed domain, recording starts automatically. No data from any other domain will be collected. The Browser Extensions produce an HTTP Archive (HAR), which is a standard format describing a web page with all the resources involved and some timings. For each resource, it has details about the request and response, with timings. This allows us to offer the waterfall view:

What Makes Up a Chrome Extension?

At the core of any Chrome Extension, there is the manifest.json file. It describes the contents of the extension and defines rules for content scripts injection (URL matchers). Content scripts are custom pieces of javascript that can be injected in users’ web pages. These insertions can be applied to specific pages.

The backbone of the app is the background.js file. It’s a singleton script that runs per Chrome instance. This script can register to Chrome APIs such as Native Messaging, Web Request and Web Navigation to allow it to monitor browser actions and communicate. This script does not have direct access to DOM and pages, but it can have access through Content Scripts. Background.js can also interact with the “popup,” which is a little piece of user interactable area that can be displayed when the user clicks on the extension’s icon next to the URL bar.

How We Leverage the Chrome Extension APIs

User sessions are made of pages linked together by the tab they are running in, their top domain and possible redirections from one to the next. So firstly, we need to get information about the open tabs in the user’s browser. The chrome.tabs Extension API does exactly that: it tells us about the open tabs, it notifies us when a tab is closing, or a new one is opening.

Then, we need to generate our HAR (HTTP Archive) structure for pages and web resources. The webNavigation and webRequest APIs are the two main pillars:

The webNavigation API gives a big-picture view of what’s going on in the user’s tab. It invokes callbacks when page navigation is initiated, when it’s committed and finally when it’s completed. We get top-level information from it, as well as timestamps we use to derive page metrics. The onCommitted event is of particular interest because it tells us that there will be no more HTTP 3XX page redirects and the browser committed to displaying content in the tab. It also provides us information on how the navigation was initiated: if it was the user (for instance the user clicked on a bookmark or typed in the address bar), or if it was a server or client redirect.

The webRequest API gets us more granular information on the resources loaded in a page. We record data on each document, script, stylesheet, ajax call involved during a page load: request information (such as url, headers), response information (status, bodySize…), timings and more.

The HAR structures put together are passed via NativeMessaging to the Agent running on the user’s device. It is its job to understand what the user journey was and workout the user sessions.

New Chrome Extension, New Tech Stack

Our new extension is written entirely in Typescript and built using webpack. You can find templates to set this up like samuelsimoes/chrome-extension-webpack-boilerplate. Typescript has many advantages that have been widely discussed. I mostly enjoy having reliable autocompletion and type checking in my frontend code.

The extension is divided into several components to separate concerns. Each component is in charge of registering its own callbacks for Chrome API events. To improve the structure and readability of the code, we have employed reactivity with RxJS. We created a utility component that defines observables for Chrome events and exposes methods to add filtering on top of that. It’s very handy to filter events per tabId for instance. Also, it allows sharing of callbacks using RxJS’ share and publish operators between the multitude of components, which can be running in parallel.

We have a very clean UI, consisting of only the Popup that appears when clicking on the Extension’s icon in the browser. We built it with VueJS because of how easy it is to set up and use. It’s the main UI Framework at ThousandEyes, nowadays.

A strong emphasis has been put on testing, and we have chosen Jest as the framework. First of all, snapshot testing has made testing the Popup Vue component a breeze. And Jest offers all the tools you need to easily deal with module loading and mocking, as well as dealing with asynchronous calls and Promises. To mock the Chrome API calls, we found jest-webextension-mock to be very handy.

Final Thoughts

We are excited for the new Endpoint Agent Chrome Extension to be released along with the new sessioning logic and the updated Endpoint views in ThousandEyes’ web app. It should prove to be very maintainable thanks to the new structure and tech stack, and it will allow us to make continuous improvements and register more data and metrics for our customers in the future.

Interested in product updates like this? Subscribe to our blog and you’ll always stay connected with product updates, outage analysis posts, and other Network Intelligence and monitoring content.

Originally published at https://blog.thousandeyes.com on July 23, 2019.