How PWAs works and how I implemented it with React and Webpack

#3 — More technical details about PWAs, Service Workers, React and Webpack

Published in

Learning Lab

14 min readAug 5, 2017

This article is part of the PWAs learning month of my Learning Lab challenge.
I will explain more about PWAs, what is the lifecycle of a Service Worker, how to debug it and measure performance, and also how I implemented it with React and Webpack for my project Kanbanote.

If you don’t know what is a PWA I invite you to read this article: https://medium.com/learning-lab/how-i-learnt-progressive-web-apps-in-a-month-and-implemented-it-635448ed1330

Advanced information about PWAs and Service Workers

PWAs are made of lot of pure-engineering optimisations

“Amazon’s calculated that a page load slowdown of just one second could cost it $1.6 billion in sales each year.”

Time is money, that is why there are a lot of optimisations in the world of web and around PWA. Those optimisations include different techniques such as:

Asynchronous rendering
Minification — Concatenation of all your files together
Route based code splitting which separates the Single Page App (SPA) in different bundle which you load separately
Use of the different caches (see Caching strategies below)
Reducing the size of images (and even having every image in different sizes for different devices)
Compression of the files
Use of HTTP/2
Managing your files priorities with the Preload, Prefetch, Async, Defer proprieties

If you are interested about this performance topic I invite you to read these two articles and check this Web launch checklist

Twitter Lite and High Performance React Progressive Web Apps at Scale — https://medium.com/@paularmstrong/twitter-lite-and-high-performance-react-progressive-web-apps-at-scale-d28a00e780a3
Preload, Prefetch and priorities in Chrome — https://medium.com/reloading/preload-prefetch-and-priorities-in-chrome-776165961bbf
Web launch checklist — https://weblaunchchecklist.com/

Lifecycle of Service Workers

A service worker is a script that runs in background of your browser separated from your website. You can use this script to cache files, send push notification or do other background task like updating your page for example.

Before starting developing it’s important to understand well the lifecycle of a Service Worker. Check as reference this simplified graph of the Lifecycle of Service Workers from Google Developers and below I will explain what happens at each step:

Life cycle of Service Workers — Source: https://developers.google.com/web/fundamentals/getting-started/primers/service-workers

Installing: In order to install the Service Worker you need to register it in your page like this.

if ('serviceWorker' in navigator) {
  navigator.serviceWorker
    .register('./service-worker.js')
    .then(function() { console.log('Service Worker Registered'); });
}

The browser will then start to install it and then downloads the statics assets you specified. If it fails downloading them all the installation fails and the Service Worker will not activate.

Activated: Then the activation event is triggered where you can handle your previous cache, delete it for example. Each Service Worker has a defined scope (the route of the website, for example “/” will give the access to the whole site) where it can act. The Service Worker will not do anything until you open another page or reload this page.

Fetch/Message: Then the fetch event will be triggered to be able to answer to the networks requests.

Here is how a simplified Service Worker looks like.

var cacheName = 'sw-version-1'; // the cache version
var filesToCache = [
  '/',
  // files you need to cache (can be anything), never cache the SW itself,
];self.addEventListener('install', function(e) {
  console.log('[ServiceWorker] Install');
  e.waitUntil(
    caches.open(cacheName).then(function(cache) {
      console.log('[ServiceWorker] Caching app shell');
      return cache.addAll(filesToCache);
    })
  );
});self.addEventListener('activate', function(e) {
  console.log('[ServiceWorker] Activate');
  e.waitUntil(
    caches.keys().then(function(keyList) {
      return Promise.all(keyList.map(function(key) {
        if (key !== cacheName) {
          console.log('[ServiceWorker] Removing old cache', key);
          return caches.delete(key);
        }
      }));
    })
  );
  /*
  return self.clients.claim();
});self.addEventListener('fetch', function(e) {
  console.log('[ServiceWorker] Fetch', e.request.url);
  e.respondWith(
    caches.match(e.request).then(function(response) {
      return response || fetch(e.request);
    })
  );
});

I you want to know more about the lifecycle of Service Workers here are two very good links:

Service Workers: an Introduction — https://developers.google.com/web/fundamentals/getting-started/primers/service-workers
The Service Worker Lifecycle — https://bitsofco.de/the-service-worker-lifecycle/

Caching strategies

The caching strategy is how you will use the cache, when to update it, and when to get the data from your network.

There are different strategies such as:

Cache only — Only using the cache and always
Cache and network at the same time — So we get the fastest response
Cache, falling back to network — It’s an offline first strategy, you first get the data from the cache and if it doesn’t work you get them from the network
Network, failing back to cache — We first try to get data from the network and if it fails we load them from the cache
Cache then network — We first use the data from the cache and then update them using the network

I invite you to check this two websites to know more about all the caching strategies and how to implement them :

The offline cookbook — https://jakearchibald.com/2014/offline-cookbook/
ServiceWorker Cookbook — https://serviceworke.rs/

How to debug

In order to debug there are two tools in Chrome, the first is the application Tab of Chrome developer tool. You have a “Service Workers” section, a section for “Manifest” and a section for all the storage. The Cache storage contains the assets cached by the Service Workers. In the Service Workers section you have different options very useful as “offline” or “update on reload” that I recommend you to use when you start.

Here you can see the debug console including the Manifest section and the different caches

The second to see all the SW installed in chrome just open a tab and open this “chrome://serviceworker-internals/”

How to mesure performance

Google Lighthouse is the tool to measure the performance of a website, its responsiveness and also its progressive web apps capabilities. It gives you a score of your website and tells you what to improve.

You can install it as a Chrome extension or even a command line tool.

Here is the result of Kanbanote 2 — Not very good as you can see

Limitations of Service Workers

The first and most important limitation of Service Workers is the compatibility, a lot of browsers are not supporting it yet. You can check at anytime the current compatibility here: https://caniuse.com/#search=service%20workers

As you can see, it’s not working in iOS devices yet. It’s more an android thing, indeed it’s google pushing toward that! And very recently Apple refused to support PWAs, read more about this here: https://m.phillydevshop.com/apples-refusal-to-support-progressive-web-apps-is-a-serious-detriment-to-future-of-the-web-e81b2be29676

When creating your Service Worker you will define required and optional files to cache, if the required files to cache fail, the Service Worker won’t install. Be careful.

How to apply it to your own project

Define what needs to be display immediately.
Define what are the key UI component.
Define what are the resources related (script, images) that you will need in the cache.
Define what is the data to cache (javascript objects), and select a storage. You can use Localstorage or IndexedDB (in the article they use Localstorage, they recommend to use the library Localforage to use easily IndexedDB as you would use Localstorage).
Define a caching strategy.
Implement it using sw-precache, a library to generate your Service Workers recommended by Google Chrome (and made by them).

Let’s implement it for Kanbanote

Now that we all know what are the PWAs and how to create a Service Worker, let’s see how I implemented it for Kanbanote.

Kanbanote is a service that I created that turns your Evernote as a Kanban board. Before creating Service Worker, and manifest file I had to make it work fully for mobile and define what are the PWA features I need.

As I describe in the preparation phase, I need to:

Implement the drag’n’drop for touch.
Use a Service Worker to cache the javascript and style files of the Single Page App.
Use IndexedDB or Local storage to cache the data to make the loading feel faster (and see the previous data before loading the data from Evernote). For this reason I want a cache first strategy.

At the time I wrote this article, Kanbanote 2 is in production and Kanbanote 3 only available in beta. You can see below how they are in term of look.

On the left Kanbanote 2, on the right Kanbanote 3 — Note that in Kanbanote 2 the dropdown to select the board wasn’t implemented, it was showing a modal to ask how much you were willing to pay to have this feature implemented

Kanbanote 2 was made of a backend in PHP, still used for Kanbanote 3, and a frontend built in angularJS. I didn’t use any dependency management tool, so I was loading them myself. To be honest my level of coding was low, so it was my code quality. Let’s see now how I evolved in Kanbanote 3.

Kanbanote 3 technologies stack

In order to implement this PWA features it’s important to understand Kanbanote’s stack and the limitation of it.

Kanbanote is using a backend made with PHP (using different modules and not a big framework). This backend has two roles. First, to handle the authentication with Evernote, once authenticated, it renders a page containing the Single Page App (SPA). The second part of the backend is just an API that is used by the SPA.

Kanbanote’s frontend is a SPA made with React, Redux, and bundled with Webpack. As you can imagine I had to redevelop the whole frontend part.

Moreover I use Circle CI to build, test and put in production easily.

If you are not familiar with those technologies I invite you to have a look on:

React — A Javascript library to build interfaces, the official React website
A cartoon intro to redux, an article which explains well what is redux
Webpack, the official website, which explains very well what it does

Also I used the Yeoman (a command line generating tool) generator fountain JS. It gave me gulp, webpack, sass, and react scaffolded, and also gave me a great command line too.

The limitations

Because I am using webpack, in order to escape the HTTP cache, the js and css files changes names everytime we build the project. That is why it’s impossible to specify the file names in the Service Worker. I then started to search how to solve this issue and I found the library Offline-plugin to generate a Service Worker for a project bundled with webpack.

Moreover, since I am using React with Redux, I looked for a library to save the whole state instead of caching manually. Also, I found a solution in the same article about React and PWA that I recommend. This library is called Redux-persist.

How to create the manifest.json

The manifest.json that contains the data to display in the splash-screen can be made easily. In order not to have any syntax issue I recommend the following very good generator.

Web App Manifest Generator

Web App Manifests are one of the key pieces to making your web app look and feel like a native app. Learn more While…

tomitm.github.io

You just have to fill the form, copy the “manifest.json”, paste it in a file called “manifest.json” and copy the content of <head> in the <head> of your main html file.

And that’s it! Then you can debug it in Chrome developer tools.

You can press the “Add to homescreen” link to test if it gives error in the console

Also try to add Kanbanote to your homescreen in Chrome mobile and you will be able to see the splashscreen.

On the left you see the icon added to my desktop like an app, and on the right the splashscreen

How to create the Service Worker for Webpack with Offline-plugin

Offline-plugin is very easy to use. First, start adding it as a dependency in your project.

npm install offline-plugin --save-dev

Second, add it in the plugins part of the webpack config file.

const OfflinePlugin = require('offline-plugin');module.exports = {
  module: {
    // ...plugins: [
      // ...
      new OfflinePlugin()
    ]// ...
  }
}

Finally, add the runtime in your main javascript entry file.

import * as OfflinePluginRuntime from 'offline-plugin/runtime';
OfflinePluginRuntime.install();

And that’s it!

Actually that wasn’t it 😅, I had a lot of problems due to the structure of my folders:

I have a folder “/public” containing my “index.php” which is served
Every time I build my project I put the whole bundle in “/public/assets”. This folder contains the “index.html” of the SPA and the app-*.js, vendor-*.js, and everything else
I had to set my public path to /assets
For some reason the manifest.json does not work if it’s not in the same folder as the sw.js (the Service Worker file generated), I had to move it to the root of the project every time I build the project using a gulp script.
Because of that I couldn’t leave the sw.js in the “assets” folder I had to move it in “public” so I had to force all the automatically cache file path to have the suffix “/assets”. I used for that the rewrite(asset) function that I defined in the options.
Last problem was that I couldn’t cache the route that is serving the SPA, because the permission is handled by the backend. If the page was cached before being authenticated it will give the home page. If it was cached after being authenticated it will always remain authenticated and the frontend may throw some errors.

You can see how my config of the plugin looks like at the end after solving these problems.

const OfflinePlugin = require('offline-plugin');module.exports = {
  module: {
    // ...plugins: [
      // ...
      new OfflinePlugin({
        // I needed this credential line to make it cache the authenticated page (that I excluded at the end)        
        ServiceWorker: {
          prefetchRequest: { credentials: 'same-origin' },
        }        
        externals: [ 
          // some images that I wanted to cache
          'app/images/favicon.png',
          'app/images/icon.png',
          'app/images/logoL.png',
        ],// the public path for the SW.js
        publicPath: '/',// the excluded files that redirects to the login page
        excludes: [ 
          '/',
          '/board'
        ],
        rewrites(asset) {
        // the rewrite to match with the real publicPath that we can see below
          return `/assets/${asset}`; 
        }
      })
    ],
output: {
  path: path.join(process.cwd(), conf.paths.dist),
  filename: '[name]-[hash].js',
  publicPath: '/assets/'
},// ...
  }
}

If you are using webpack I fully recommend you this plugin!

NekR/offline-plugin

offline-plugin — Offline plugin (ServiceWorker, AppCache) for webpack (http://webpack.github.io/)

github.com

How to implement Redux Persist

Redux Persist is also very easy to use. First, add redux persist as a dependency

npm install redux-persist --save

Then, add these lines to your store

import {persistStore, autoRehydrate} from 'redux-persist';const store = createStore(
    reducer,
    // ...
    compose(
    applyMiddleware(
      ...middlewares
    ),
    autoRehydrate() // add this line
  )
);persistStore(store) // and this one

And that’s it! Every time your state will be updated it will be persisted in the localstorage. Once you reload the project the latest state saved in your localstorage will be loaded, it works very well.

But again, that wasn’t it 😂, of course! I had a few problems.

It’s not a real problem, just that one of my goals was to save the state using the IndexedDB and not the localstorage. This is very easy, you have to load the “localforage” dependency to your project, import it in the file and set when running “persistStore()”.
In Kanbanote’s navbar there is a “syncing” spinner, rotating when an API call is running. Basically it’s a variable from the state which is incremented when the call start, and decremented when the calls is ended or fails. The problem was that the state of this icon was also saved and even though when I load the project I always reset its value, it didn’t work. The reason is the asynchronous way of working: when I launch the app the state is empty, and then Redux-persist will put the latest value saved of it. This was actually easy to fix, there is a way to blacklist the different reducers of your store. But this only works when the reducers are well separated, and that couldn’t solve my next problem.
For the same reason of asynchronous, I had another issue. Let’s imagine my Kanban contains two different Boards, A and B. The default board is always the first, so A. If you load Kanbanote for the first time it will open this default board, so the A. If you open B and close Kanbanote, and reopen it. The last state will contain B, and then the API call will call the default board. Because of asynchronous I couldn’t get the value of the latest board before doing the call otherwise I could ask to load the data of the right board. After struggling a lot I saved this problem by saving the latest board opened in a cookie of the backend.

In order to solve 1, and 2 here is how my code looks like.

import {persistStore, autoRehydrate} from 'redux-persist';
import localForage from 'localforage'; // to save the data in IndexedDBconst store = createStore(
    reducer,
    // ...
    compose(
    applyMiddleware(
      ...middlewares
    ),
    autoRehydrate()
  )
);// The storage changed to IndexedDB thanks to localForage and the reducers that are blacklisted
persistStore(store, {storage: localForage, blacklist: ['sync', 'errors']});

So I spent a lot of time to understand and solve this last 3rd issue and at the end I somehow made it, it works, not perfectly but good enough to be used and deployed to production.

I also recommend this library which is very good when your state is well defined.

rt2zz/redux-persist

redux-persist — persist and rehydrate a redux store