Offline POSTs in Progressive Web Apps
As the possibilities with Progressive Web Applications (PWAs) are widening, it’s adoption for real world apps with huge user base has been increasing rapidly. Few PWAs to be mentioned are Twitter Lite, Alibaba & Flipkart Lite.
We @ Mobisy are building the next generation of mobile first sales automation products as PWAs. The client app is built using React, Redux, Material UI, Babel (for compiling), JavaScript (ES2015 and beyond), Webpack 2 (for bundling and chunks), Yarn (for modules), Web App manifest (for app install, launch style and splash screen). To achieve better launch time and performance we adhere to PRPL pattern.
The prime requirement for our apps is to be usable even when device connectivity is slow, unreliable or absent. These conditions are real for our 100k+ users who are sales executive and visit retailers across India. To run the app in above scenarios we use Service Workers, sw-precache (static content), sw-toolbox (runtime caching), Dexie (IndexedDB wrapper) and a thumb rule to make every component work only with offline storage. Once you login to our app and sync data, you are good to go offline.
But wait, does our app runs in view only mode? NO, we have a mechanism to make POST requests as well when offline. There are other approaches like PouchDB and Background Sync to achieve the same but we wanted a client only solution.
Storing your POST requests in IndexedDB and calling fetch when network is available is the straight forward solution. But what if you a scenario where you have series of interdependent POST requests i.e response of one/many requests is used to create the payload for subsequent requests.
To set the context and explain our solution we will take the following workflow as example: A salesman for a FMCG company discovers a new lead (retailer), creates the lead in system, schedules meeting with the lead right away and places the first sample order using the app at an interval of few hours at a location with no connectivity or his device data plan fully consumed.
The POST calls to schedule meetings and order APIs require the lead to be created first.
Each entity has a table in local storage and component use it as their data model.
Solution:
We use the Service Worker (SW) and the indexedDb to execute the requests at regular interval and when the device comes online.
Each component will register their requests, to be executed by SW, with payload or without it if it cannot create one at that moment because of dependency on some other requests. Each registered request has a uuid and a sequence number. SW will pick all POST requests, make calls based on the combination of uuid and sequence number either to server or back to the component via registered callbacks if the payload is missing. In case component’s create payload callback is invoked, component should check if it’s dependencies have already been created, create the payload and update the POST request submitted by it. The two way communication between app and Service Worker is via messages for which we built a message handler utility.
For the app to completely work offline the components’ data model should be present in local storage (indexedDb) and any updates expected out of POST request success should be performed first on the local data model as soon as request is registered.
App components would go through following steps using the common components to POST requests(Refer Fig 1):
1. FormDataStore- Store the raw data created for an entity in the app. e.g Lead, Meeting & order create pages. Along with the form data, uuid and a sequence number will also be stored.
The store API should return a uuid to the component to be used in subsequent call. This uuid helps to associate the form data, callback and POST requests.
For our example, meeting can be created for a lead which was created offline and no real lead id is created yet. The form data for meeting can be stored with the uuid of lead and a sequence number 1.
2. CallbackRegistry- Each component which uses the POST mechanism has to register a callback class with this registry. The callback would have 2 functions- One to process the POST success with response from server, other to create payload and update the POST request table when requested by SW.
For our sample scenario, as lead is a top level entity, we need not have create payload callback implemented. The POST success callback should update the locally created lead.
In case of meeting component, the create payload handler should peek the formdata, check if the lead uuid has been been successfully posted by checking if a real id is assigned, create and update the POST payload for this specific meeting request.
3. POST Requests- All the POST requests from app should be stored in local storage with a uuid & sequence number. Requests having dependency on other request should be using same uuid and set the sequence number higher than what currently exists in request table. Also, for dependent request do not store the payload in the request table. This would ensure that Service Worker would call the request originator when it’s POST request is inline, instead of calling the server.
The following fallback mechanism needs to be in place as well:
1. Have retry mechanism in SW for all pending requests at regular intervals if their status is failed.
2. App sends message to SW on network state changed to online and the POST retries are triggered.
3. Use the client logging framework capture the pending POSTs requests and related details to monitor, in case the requests are stuck for long.
The above solution is already powering our first enterprise offline PWA in production and we are confident of releasing the next ones on same platform.
