Caching REST calls when offline

Some members of the OpenPaas team and Linshare team had a meeting in Agde for a desktop bar-camp. The aim was to produce a desktop version of OpenPaas using Electron that will also support offline mode. While a part of the team was working on packaging OpenPaas into Electron the rest of the team has been working on offline support.
The first use case is to let the user organize his emails and his calendars when offline.
All the actions done offline are saved and sent to the server once the client regain network connection.

Architecture of the solution

Firstly, we wanted a solution that could be turned on and off (feature flipping) and that did not ask for a lot of modifications on the back-end part. The only requirement of the back-end is to detect concurrent modification. This is already the case for event (thanks to E-Tag of WebDav) and for email (thanks to ifInState of JMAP spec).

Our main idea was to add a generic angular service, offlineApi, to ease the support of offline mode for OpenPaas module. This service responsibility is to store actions when the user is offline and send them to the server when the user regains connection. It should also notify module that use the offlineApi of the actions’ state.

Each module that wants to support offline mode needs to register all the different actions which need to interact with the server. This is done using:

offlineApi.registerActionHandler(
  module: string,
  action: string,
  callback: localRecord -> restangular result
)

The type of local record will be defined below. The module parameter is to avoid collisions across module that might want to register actions with the same name. The action parameter identify a kind of action, for example “create an event”, “remove an event”. The callback will be called with all the necessary informations to perform the action by the local record passed as an argument.

The local record is an object with the following fields:

• module: string
• action: string
• payload: object, it can be anything and it is defined by the module when calling offlineApi.recordAction.

When the module have recorded all the different kinds of action he needs, instead of directly doing a request to the server, it can now call offlineApi.recordAction by passing a local record. The module can use the payload field to pass all necessary informations to perform the action on the server. For example, in the case of an event creation this could be a jCal object.

When offlineApi.recordAction is called, our module will check if we are offline or online:

• if we are offline, the callback registered before will not be called yet and the local record will be stored on the browser using localForage. Once the network is back, we will fire the callback on it.
• if we are online, we will also register the local record in case we get a time-out exception (offline situation that has not been detected). Then we fire the callback and only remove the local record on success or error but not on time-out error.

The offlineApi.recordAction returns a promise that resolves with the localRecord after adding an id to it so we can easily delete it later.

Offline detection

At the beginning, we had the idea to use window.navigator.online, but it turn out to be a bad idea because the definition of offline is not the same across browser and at best it detects offline when the network interface is down!
So we got the idea to take advantage of the websocket already in place on OpenPaas, in fact websocket connection have two interesting events:

• onConnect
• onDisconnect

It turned out to work pretty well during our experimentations.

Optimisation

An example of place optimisation

We have also added a listActions method to our offlineApi that allows us to list all the actions waiting for connection. The idea is that sometime, a new action can replace an old action. This brings the opportunity for a module to inspect its own actions and remove them with the help of offlineApi.removeAction.

For example, in the case of the calendar module, any modification or deletion on an event removes all previous pending creations or modifications about this event. So when the user regains connection instead of doing two sequential requests, we merge them into one that will have the same effect.

The offlineApi.listActions is also really important if the web application get refreshed when offline. The module needs to be able to know unfinished operations from previous session. For example, in the calendar when reloading the web application we look for all pending creation/deletion to display them correctly.

Conclusion

During the bar-camp, we were able to add offline support for CRUD operations on event for the calendar module.

The most difficult part was to consider that the feedback of a CRUD operation on the server does not come anymore from a promise but from a callback that can be called potentially an hour or even some days later, after a total refresh of the application (reboot of a laptop after a flight for example).

You then need to be able to pretend the action has been done and to be prepared to revert it at any time.

Working on offline support is something challenging and it was really enjoyable.