Testing Service Workers

In this post we’ll be looking at some of the approaches being taken to test service workers in the latest service worker libraries the Chrome DevRel team have been working on.

The goal of this document is to provide some practical examples and methodologies for testing service workers that you can take away and apply in a fashion that works for your project and team.

Regardless of your testing requirement — whether it’s ensuring your service worker is caching the assets you’d expect, checking your site is returning valid responses when offline or you want to test one of the available events like push — there should be something in this article to that guides you through approaches of testing your code.

Types of Testing

There are a two ways to approach this topic of testing service workers. We could look at testing specific features or use-cases, for example testing a web app works offline. The alternative is to look at some of the methodologies for testing service workers, for example looking at how we write a unit test in a service worker environment, regardless of what that unit test is testing.

In this post I’m going to look at the methodologies. Looking at techniques should enable us to realise how we can apply them to any API and feature associated with service workers, whether it’s offline caching, push events, background sync or a future API that hasn’t been implemented yet.

Service worker testing can be dissected into common testing groups. At the top level we have integration tests. These are high level tests that examine overall behavior. For example a test could load a page, register a service worker, stop the server (creating a pseudo offline state), refresh the page and makes sure the page is loaded from the service workers cache. These tests won’t be fast and may fail intermittently due to webdriver issues and/or race conditions.

The next level down would be running unit tests in the context of the browser. These are tests that could register a service worker and examine the results. An example test could include registering a service worker, wait for it to install, check that a specific cache was created with specific set of assets cached. These tests aren’t particularly fast, but the development cycle in the browser is generally faster than integration tests.

Going another level down we can write unit tests in the context of a service worker. These tests are specifically useful when you have functions that make use of the JavaScript API’s that are only available in the service worker. This normally requires your logic be fairly self contained. An example unit test could call a function that handles showing a notification in response to a push event: the first call to this function should open a notification, and the second call to the function should replace the first notification with a new notification with some collapsed text.

Arguably all of these tests are integration tests since they require running in an actual browser, which in itself introduces a level of pain to automate. The final “level” of testing is to fake the API’s you are trying to test. This leads to more reliable and faster set of tests. The only real risk is that the mocks can fail to match browser behavior.

All of these tests have an appropriate time and place and you should consider how and when a test makes sense to have for your project. There’s no point of having integration tests if they aren’t run on a regular basis.

The order we’ll be looking at these testing methodologies are:

1. Browser Unit Tests
2. Service Worker Units Tests
3. Integration Tests
4. Mock Environments

The reason for this flow is that it’s slightly more natural to start off in the browser and then step back out of the browser.

Tooling

Throughout this post I’ll be using MochaJS, this is by no-means a requirement for testing service workers, nor is this a suggestion that this is the best tool for the job, but it’s easy to comprehend and you should be able to apply the techniques used with MochaJS to your prefered test runner, if you can’t . . . . maybe try Mocha.

Browser Unit Tests

Mocha JS has support for writing tests in the browser, you’d typically create a HTML page, add script tags for mocha itself as well as script tags for your test files. Create a new html file and save it under ‘/test/browser/index.html’ in your project.

If we load these page into our browser, we’ll see the following UI:

Super exciting huh?

From this we can start to write a basic service worker test.

We’ll create a new file and in put it under ‘/test/browser/first-browser-test.js’. When adding new tests, you’ll find helpful to group “kinds” of tests in folders helpful (i.e. all browser tests under /test/browser/ and all service worker tests under /test/sw/ etc).

If we add this new file to our html page, like so:

We’ll get the tests showing up and running in our page.

Time to fill out our unit test with some code that’ll actually use our service worker. First we need to register a service worker.

If all we do is write this and run it, we’ll get a failing test because `/test/static/my-first-sw.js` doesn’t exist.

This is already showing some worth. If you had a service worker in your web app, you can see how a simple test would catch if anything was wrong with the file / registration.

We can create an empty service worker file at ‘/test/static/my-first-sw.js’ and we’ll get a passing test again.

Next let’s address the “Wait for service worker to install” step from our unit test instructions.

The reason we want to wait for the service worker to install is that if we want to check that certain files were cached, we need to wait until the service worker has “installed” as that’s the service workers signal that all required files have been cached, meaning our tests can perform its checks.

For this we can write a small utility function to detect when the service worker has changed state.

What we need to do is get the service worker being installed and then add a statechange event listener so we can detect when the service worker has been installed (or any other state). Then we remove the event listener from the service worker statechange event and either resolve or reject our returned promise.

This can be saved to a file and added to our index.html page.

With this we can tackle the second step of our unit test — waiting for the service worker to install.

But alas, we have a failing test again.

What is going on? Why the error “The service worker is not installing. Is the test environment clean?” coming up?

If you clear all your service workers in DevTools and run the test, it’ll pass, refresh the page and suddenly this error comes back, what gives? Service workers by design, once registered, stay registered, meaning if it’s installed, it won’t install again until either the service worker file changes or it’s unregistered. The fix is to ensure between tests any existing service workers are unregistered.

Let’s start off with writing some code that makes sure all service workers are un-registered before we run our tests. Inside our `describe` call, we can use Mocha’s “beforeEach()” function to unregister all service workers before each test is run.

What this does is say, “before every test, run this function” and in that function we are getting all the service workers and unregistering them. This way every new test will be starting in a fresh state.

Refresh the page and our tests are passing again. The final step is to check if the install step successfully cached an expected request. First we need to make our service worker cache something in it’s install event.

Let’s add some pretend data to a cache in our ‘my-first-sw.js’ file.

Now we need to check it’s working as intended in our test case.

That’s it, we can now check that our service worker registered, it successfully installed and ensure its caching a required request!

There is one problem with this — we are actually leaving the cache populated between each test run, this could result in passing tests when it should have failed.

We’ll move our service worker unregistering to a helper method and add cache clean up to it as well.

We save this to a file `/test/utils/sw-test-cleanup.js` and add it to your `/test/browser/index.html` file.

And finally our tests can use this before and after each test.

With this our tests will be starting from a fresh state again and we should have reliable tests.

This approach to testing is nice as it feels similar to how you’d use service workers in your web app. You’d register it in your web page, it would perform some useful action (in this case cache some requests during on install) and after that the browser will use your service worker when needed.

Next we’re going to look at writing unit tests in the service worker environment. This can be helpful for testing piece of logic your service worker or test API’s that are only available in a service worker environment.

Unit Tests in Service Worker

Mocha’s API is happy working in a normal JavaScript worker so it’s easy enough to get it working in a service worker.

Running unit tests in the service worker itself is helpful since as it allows tests to use API’s available in a service worker.

To set up our service worker tests the process we need to take are:

1. In our web page mocha tests, add a single unit test that registers a service worker containing unit tests.
2. Send a message to the service worker to initiate the tests in the service worker.
3. Wait for message from the service worker with test results
4. Pass or fail the test in the window.

In our `test/browser/index.html` add a new script tag for `/test/browser/run-sw-tests.js`:

In this new file we can scaffold out a Mocha test that will be responsible for running the service worker tests.

Step 1 is to register a service worker file containing our service worker unit tests. Create an empty service worker file in `/test/sw/sw-unit-tests.js`, then from our browser test, register the new service worker, not forgetting to cleanup between tests.

With our service worker registered the next natural step is to write a test in `/test/sw/sw-unit-tests.js`.

To use Mocha in your service worker you can import mocha.js using importScripts(). With this we can configure mocha in a similar fashion as we’ve done in our index.html file, the only difference is that you need to set the reporter to null (this prevents Mocha from trying to write to a DOM element).

If you run this code, you’ll be able to see a our log message printing the results (along with logs from Mocha) but the result doesn’t actually get transferred back to the browser, this means that if our service worker test fails, the browser test will still pass, so it’s not clear whether all our tests are passing or failing.

To account for failures in our service worker tests we’ll need to alter our browser tests to message the service worker to start running the tests and wait for a response from the service worker with the test results.

In our browser tests we can write a function that sends a message to a service worker and waits for a response:

In our test, we can send a message after we’ve registered `/test/sw/sw-unit-tests.js` and then throw an error if any of our service worker tests have failed.

The final step is to alter our service worker so that we wait for a message before starting to run the tests and then return the results once the tests have finished.

This way, any failing tests in our service worker will fail the browser test.

Up to this point we have a few things covered:

• We have unit tests running in the browser context.
• We have unit tests that register and tests some behaviours of service workers as a by-product.
• We have unit tests running directly in a service worker context.

With this you can test code in the browser window and you can test code in the service worker.

One thing that is not obvious but can be useful in service worker unit tests is the ability to emulate events.

Emulating Service Worker Events

From within a service worker you can dispatch fake events by simply constructing a new event and dispatching it.

Constructing a “fake event” is simple you just need to use the correct constructor for the type of event you want to dispatch. For a push event we could create an event like so:

With this we can dispatch an event in our service worker like so:

To make this easier to test we can override the `waitUntil()` method on the event, this allows us to detect when the returned promise is complete before testing the result of the event.

All this put together means we can create a test case that checks a specific notification was shown like so:

While this example is trivial, it hopefully demonstrates an easy way to orchestrate more elaborate scenarios like unexpected / bad data in the push messages data or perhaps ensuring that notifications exhibit specific behavior (Like collapsing multiple messages into a single message).

We can perform the same kinds of tests for ‘fetch’, ‘notificationclick’ and ‘notificationclose’ events.

Fetch Event

The main difference to note here is that depending on what you are trying to test, you may want to override event.respondWidth instead of event.waitUntil — this allows you to test the response that would be returned to the browser.

Notification Click Event

A common use case for testing notification click would be to ensure that the notification that was clicked was closed and perhaps ensure that certain actions were taken (like tracking the event with analytics or opening a window).

Notification Close Event

A Note of Test Structure

We’ve seen that with emulated events we can write tests that will run through what a browser would do in a service worker, but there are a few things to highlight.

If you want to go through this path of testing, it’s a good idea to put your event listeners in a separate javascript file and add it to your test file using importScripts(). This gives you the ability to use these event files in your production service worker while also using the same code in your tests.

For example, if we altered our push notification test above, we could create a file `/test/static/example-push-listener.js` and import it in our tests at the top of our file with `importScripts(‘/test/static/example-push-listener.js’);`.

The advantage of this is that in your actual site we can import and use the exact same file.

The alternative approach to all of this is to skip triggering the events but instead restructure the code into standalone function or class and test that method directly. This removes the need for fake events altogether, the only risk is if the events are hooked up to this logic incorrectly (which feels like a minor problem).

To give an example of this, we could have a simple push event like this:

We can rewrite the code to be a function that expects an event as an argument:

Our tests would then call `self._handlePushEvent()` directly instead of dispatching a fake push event.

In our production service worker we’d write our push event listener like so:

With this, you can start to see how we can build up logic and code that can be used in our service workers without having to worry about when and how it’s used in the service worker (i.e. remove the dependency on events).

Note: When using importScripts() in production, please make sure your imported files are revisioned in the filename so that if the content changes, the change in the service worker will trigger a service worker update (meaning your users will always see the latest service worker and import scripts). Instead of `importScripts(‘/scripts/fetch-manager.js’)`, use a tool or server side logic to add revision detail like `importScripts(‘/scripts/fetch-manager.1234.js’)`.

At the moment, I don’t think there is a hard and fast rule for when you should have unit tests in the browser vs in the service worker environment. Personally, writing a meaningful and useful test is the goal, where it lives is up to personal preference.

Real Service Worker Events

If you’ve worked with service workers for caching and responding to fetch events, the fake events can feel inadequate and / or unnatural compared to how the service worker is actually used by the browser.

There are techniques you can use to fully test an end-to-end flow for service worker events like fetch and service. Depending on what you are trying to test will determine how you can test those events.

Real Fetch Events

First let’s look at how to test fetch events in a manner that includes the full browser behavior of using the service worker as a proxy for network requests.

The biggest issue with this kind of testing is that it requires a web page to be controlled by a service worker, meaning that all requests are going through the service worker. We don’t want our browser test page to pass all requests through a service worker as it’ll cause confusion if tests start to behave different as a result of the service worker intercepting requests.

The best approach I’ve found to test service worker fetch events with a test runner like Mocha, is to take advantage of service worker scopes to scope a service worker to a unique page. This ensures each test is self contained and we have reliable pass and fail results.

If you are new to service worker scopes, here’s the brief 101.

Service Worker Scope 101

When you register a service worker, it is given a “scope” which be default is the location of the service worker. The scope restricts which pages are controlled by a service worker.

For example, if we register a service worker with this code:

What we are saying is that the service worker should only control pages that have a URL starting with /blog/ on our site. If we left out the scope argument, the scope would have be ‘/’ since that is the location of the service worker file, in that case any page starting with ‘/’ will be controlled — i.e. every page on the site.

One thing to note is that scopes must can be made longer that the location of the service worker file, but they can’t be shortened or completely different.

There is however a way around this limitation, if the web server that serves the service worker file includes a ‘Service-Worker-Allowed’ header, the header value can define the valid minimum scope.

Real Fetch Events + Zombie Service Workers

What does “scope” have to do with testing fetch events?

Well, turns out that registering a service worker, un-registering that service worker and re-registering the same service worker results in what I’ve affectionately named the “zombie service worker”.

A zombie service worker is basically the result of a browser un-registering a service worker (i.e. marking it to be killed and removed) but then bringing that same service worker back to life when the same service worker is registered again. When this happens, the service worker won’t go through the normal lifecycle of “installing”, “waiting” and “activating”. Instead it gets in a funky state.

With scoping, we can register a service worker again a unique scope meaning that we can un-register and re-register a service worker as many times as we want and avoid zombies (i.e. unique scopes means unique service worker instances).

We can take the following steps in a single unit test:

1. Register a service worker with a unique scope, in recent projects I’ve just been using ‘/test/iframe/<Timestamp>’, but it can be anything as long as it’s unique for each service worker.
2. Add an iframe to the page.
3. Set the iframe’s src to the scope from step 1 (i.e. iframe.src = ‘/test/iframe/<Timestamp>’).
4. In your unit test, get a reference to the iframe in your JavaScript and use `iframeElement.contentWindow.fetch()`. This will make network requests through the iframe’s page, which will be controlled by the service worker, meaning your request will go through the service worker.
5. After each test remove any iframes that were created and unregister any service workers.

This feels hacky, no doubt about it, but it has the advantage of restricting / containing the behaviour of each test.

In terms of implementation details, I needed to make the test server respond to ‘/test/iframe/<Timestamp>’, which can be done with express js like so:

To add the ‘Service-Worker-Allowed’ header on the test server we can add some middleware that will add the header to every request — this allows up to put our service workers anywhere and scope them to any path we wish.

I’m on the fence as to whether this should be recommended way of testing service workers. Originally I went through this hassle to test service workers due to browser differences in the early implementations. Being able to test the full flow was helpful to account for any and all quirks and behaviors. As service workers become more stable and widely supported this kind of testing will become less relevant and hopefully alternative approaches will be sufficient.

Real Push Events

We’ve seen how fetch events can be tested with a scoping and iframe dance.

To test push notifications from end-to-end, there is no need for scoping, but there is a need to automate granting permission to show notification.

We can do this for Chrome and Firefox with WebDriver. We need to configure each browsers profile to grants the permission (either for testing or for a specific URL).

For Firefox we can enable the notification permission for tests like so:

For Chrome we can set up a profile for a test browser that enables notifications for a local test server origin (Note this approach also works for Opera as well).

With this, you can write unit tests that subscribe to push notifications and use the PushSubscription and to trigger push messages.

In the web-push-testing-service I check if a push message is received by waiting for a message from the service worker. In my browser page (this could be in the unit test) I add a message listener like so:

And the service worker posts a message whenever a push message is received.

There are other options to validate that everything worked, but it’ll depend on what you are trying to test and what is easiest for you. For example, you could wait until getNotifications() returns with a notification, indicating that a push message was received and a notification was shown as a result.

Mock Service Workers

The last thing topic to discuss is arguably the easiest approach for testing, mock out the service worker API’s and test on node.

When you’re writing unit tests, it’s preferable to make them as fast as possible to encourage regular running of the test suite during development. Mocking out service worker environments means no browser spin up cost, no web requests and ultimately a bare bones approach to test your code in a predictable manner.

The only negative of testing against a mock is that if the mock’s API surface is incorrect it won’t be discovered until manual or integration testing (although this is easily fixed) and it assumes all browsers behave the same.

At the time of writing I wasn’t able to find any existing mocks for service worker APIs, but it’s easy enough to write your own mocks that kick the tires of your code. [NOTE: After publishing this article Zack Argyle published the service worker mocks he’s been using for testing — looks promising]

One way I’ve testing code is to create a “self” variable between unit tests which has specific API’s mocked out.

This is very lightweight approach and easy to understand when writing your tests.

Let’s say I had a library that was designed to run in service workers and calling a method `setUpEventListeners()` should register event listeners for install, activate and fetch. We can write a unit test like so:

This approach has a wide range of pro’s and con’s. Super fast but ignores real browser behavior.

Closing Remarks

A lot of the challenges you will face in testing initially is setting up the testing environment. Hopefully this will be less relevant over time through the use of mocks and test runners evolving to support unit tests in multiple environments without needing configuration from consumers of the tools.

While most of the popular testing frameworks don’t have plugins that support service worker unit testing (Hence the custom work outlined in this article), there are some tools exploring this space. The Karma project has helpers for running tests in the service worker context using Mocha and Jasmine test runners.

Hopefully you can see a path to explore your own testing and see what will work best for you. All of the code from this post is available on Github under the web-testing-examples and it includes some basic examples of running everything from the command line as well as testing a basic page.

Thank you to Steve Workman and Jeff Posnick for reviewing and thank you to Steve for calling out additional tools / libraries.

Thank you to Zach Argyle for reaching out regarding the service worker tools he’s been working on.