Offline support for Symfony applications.

In modern web applications, providing offline support has become increasingly important. This is because users expect to be able to access an application even when they are not connected to the internet. However, implementing offline support is not a simple task as web applications are designed to work in a connected environment.

Most web applications rely on a server to provide data and functionality to the client. When a client goes offline, the application loses its connection to the server and can no longer access the data it needs. As a result, web developers must find a way to store data and functionality locally so that it can be accessed even when the client is offline.

In addition, implementing offline support can be complicated because it requires developers to consider various factors such as data synchronization, data storage, and data security. The process involves determining which data should be cached for offline use, how to synchronize cached data with the server when the client goes back online, and how to ensure that cached data is secure.

Despite the complexity involved in implementing offline support, it is essential for modern web applications. By providing offline support, developers can ensure that users can continue to use an application even when they are not connected to the internet, which can improve user engagement and satisfaction.

for one of our clients, after releasing the first iterations of his platform using Symfony and Symfony UX beauty ( Stimulus, Turbo ), they noticed that the offline mode is a must-have to ensure that operators that are working sometimes in some zones where we lose connexion or in parking to be able to fill some checklists even without connexion.

How can we support Offline mode?

There is no big documentation on the internet to figure out the best options and patterns to consider. so what you will discover in this Article is an architecture based on our experience and our client’s needs, I’m totally open to discussing it if you have better ideas.

let’s assume we build a todo application in Symfony and we want it to support Offline.

the first task is to split the problem into small problems and ask the following questions:

• can we submit a POST request to the backend when the connexion is lost?
• when connexion is lost, where to store submitted forms?
• if we consider storing data locally, how can we sync it with the backend when the connexion is back?
• if our form contains files like uploading images how to deal with them?
• if someone else pushed something to the backend while we are offline, how to retrieve and sync this data?

I know, it’s a lot of questions, but step by step, the vision will be clear …

1 — Navigate without connexion ( Hello ReactJs )

the first thing to do is to be sure that we can post data and navigate if we have multistep forms, without doing any requests to the backend, to avoid reloading the application and loading the page.

to do so we used ReactJs. On the page where we need the offline mode. we can just create a react app using ( Webpack Encore ) and attach it to a div.

This react app will host our form or our multistep form and will do all the fronted parts while the connexion is lost. if we have the react app loaded we can imagine a form with the next button that changes the state to show the next part of the form etc.

2 — Where to store data ( Hello Couchdb / Pouchdb )

Now that we have a fully functional react application that we can use to fill a multistep form, and use Dropzone for example to add some attachments.

when we click to submit, where we should store this data?

at this stage, we leveraged the power of pouchdb. the definition from the PouchDB website:

It enables applications to store data locally while offline, then synchronize it with CouchDB and compatible servers when the application is back online, keeping the user’s data in sync no matter where they next login.

With PouchDb ( https://github.com/ArnoSaine/react-pouchdb ) we are now able to communicate with a CouchDB instance, and all the heavy lifting of synchronization and replication is done automatically. Also, PouchDb makes it very simple to query data and manipulate it ( CRUD operation ) and filter using hooks and using React components see the example below

import { Suspense } from "react";
import { PouchDB, Find } from "react-pouchdb";
<PouchDB name="dbname">
  <Suspense fallback="loading...">
    <Find
      selector={{
        name: { $gte: null }
      }}
      sort={["name"]}
      children={({ db, docs }) => (
        <ul>
          {docs.map(doc => (
            <li key={doc._id}>
              {doc.name}
              <button onClick={() => db.remove(doc)}>Remove</button>
            </li>
          ))}
        </ul>
      )}
    />
  </Suspense>
</PouchDB>;

the idea is now a little bit clear.

when the user visits the page where we initialize the react application, automatic calls are done by PouchDb to sync data from and to CouchDB to push new objects stored locally if there are some and to retrieve objects stored in CouchDB if there are some changes in the distant CouchDB instance.

What about attachments and file uploads?

CouchDB has a great feature to manage attachments of an object, with a special key _attachments in the object we store with PouchDb we are able to add multiple files as attachments encoded in base64 and CouchDB handle them perfectly.

see ( https://pouchdb.com/guides/attachments.html#image-attachments )

note that Pouchdb uses IndexDB locally and the big difference between storage engines like WebSQL/IndexedDB and the older localStorage API is that you can stuff a lot more data in it.

From Relational database ( Mariadb ) to PouchDb?

We have now a React application that can store data locally and interact with a distant database instance (CouchDB), it can Synchronize and replicate and also manage conflicts using PouchDB. A big part of the offline process is done.

but in our scenario, the users can add some configurations, or some data using regular Symfony forms and workflow. and this data is required in the offline application. so how to push this data to the CouchDB instance?

From Mysql to CouchDB: this is for me the easy part, and Symfony has all the tools to handle this part. in our case, we’ve just used Symfony Messenger to dispatch a message in the queue each time an update or creation of this data is done. this message is handled by a worker that interacts with CouchDb using the CouchDb SDK.

From CouchDB to Mysql: for now, we are using the cron job which is also just a simple Symfony command that uses the Couchdb SDK and fetch the latest changes and save the state object pulled in the database. but with the release of the new Component Webhook, we are planning to use it with webhooks provided by CouchDB to get the updates in real-time and handle them using Messenger ( this is work after the upgrade of Symfony )

small schema to illustrate all the processes: