What we learned from data persistence in our growing Electron app

At Station, we are building an app based on Electron. To handle the state of the application — which contains the list of installed apps, user settings, and more — we use the well-known redux framework.

Basically, whenever a user re-opens Station, we want them to recover their previous configuration. Which meant we needed to find a way to make the redux state persist.

As simple as redux-persist

At first, we leveraged redux-persist with a simple local-storage backend. The redux-persist regularly serializes the store and dumps it into the DOM local storage. Whenever the redux app reboots, redux-persist gets the data from the local-storage and rehydrates it into the store.

It’s simple and it got the job done. At first.

We recently had to implement a support for multiple windows in Station. This feature led us to share the redux state of our application across different windows, and thus across Electron processes (we used electron-redux but we’ll leave that for another story 😉). With this architecture, we had to manage data persistence in only one of these processes. Electron’s main process was the obvious choice. However, since Electron’s main process has no access to local storage, we were going to need an alternative.

This was the perfect opportunity to settle down, take a breath (we’d been pretty much shipping new features and new versions for 6 months straight!), and think about how we should go about persisting data in a growing Electron application 😌.

The main issue with redux-persist was that we had to keep our codebase retro-compatible with the old state versions of the store. For instance, if, in one of the reducers, we’d rename a key, we would have to deal with the 2 versions of this key in our selectors. Our first move was to use redux-persist-migrate, adding a migration step in redux-persist.

Again, redux-persist-migrate got the job done, but we weren’t totally happy with it. Indeed, without a clear data schema definition, it was easy to lose track of what needed to be migrated. Version after version and across devices, we accumulated data structures that weren’t what we expected them to be in the newest version. And as a result, our code base was riddled with inelegant safety-checks on various inconsistencies (missing keys, etc.) which were rapidly making the whole thing unmaintainable.

Persist data deserves better

We all agreed that we had to find a better solution for data persistence, and our requirements were clear:

• we needed to put a layer of abstraction between the persisted data and the redux store. Therefore 1/ we can be very flexible with redux store (change reducers keys etc..) without worrying about persistence and 2/ there is a clear interface between the redux store and the persisted data.
• we need to have a strict data schema for our persisted data. Therefore we know for sure that data coming from persistence respects a certain schema and we don’t have to manage several data schema in the app
• as a consequence of having a strict data schema, we wanted the migration process to be solid

We realised that we were facing the same problematics as backend engineers have dealt with for years: we need to persist data on disk in a maintainable / accessible manner and we need an interface to access it. We actually need a database and an API… but in an Electron application!

The rigor of SQLite to the rescue

For database, our choice was restricted to the embeddable databases since it had to be packaged and run in an Electron application. The classical MongoDB or MySQL were not among our possibilities.

In opposite to regular databases, the possibilities are narrower for Electron-embeddable databases: nedb is the equivalent of MongoDB while SQLite is the relational database of choice for many embedded systems. There are also basic storage solutions like node-localstorage that imitates browser’s local storage API and is backed by a JSON file.

Given our requirements, and especially our need for strict data structure in persisted data, our choice was quickly guided to SQLite.

However, using SQLite in Electron is a tad more complex that installing whatever JS npm module because it is essentially a native nodeJS dependency (a module that includes C/C++ code that has be compiled towards the target platform 💻). Fortunately, electron-builder handled that for us. But, in the process, we lost the multi-platform build and are now forced to build the Windows version on Windows and the MacOS version on MacOS.

We were definitely able to use sequelize as ORM for the SQLite database. For the migration part we used the sister library umzug. Every pending migrations is run when the app starts.

Now, when the state is modified, we summon our sequelize API, which is in charge of computing what needs to be saved and how.

We have been using SQLite as persistence backend for few weeks now. And with SQLite we gained a whole lot more confidence in the development of new features.

Here is some advice if you plan to grow an Electron application that persists data on disk:

• 🤔 Early on, define an API between the persisted data and the redux’s state. It will ease following refactoring. You can rely on redux-persist as it can be extended.
• 🎓 If your persisted data model gets complex, you’ll probably need SQLite: don’t be shy, dive into it, you’ll get rigor and confidence from using it.

In the future, we plan to make Station load normally from wherever you connect from by storing its state in the cloud ☁️. The fact that we now have a clearly defined API between our redux store and our persistence backend will make this feature possible. To be continued 🚀 !

This story was cooked with the entire Station team (Alexandre Lacheze, Mathieu Débit and Julien Berthomier). Don’t miss the opportunity to try the future of work and ask an early access on getstation.com — or read the story behind Station here.

If you liked this story, you might love working with us on Station. Don’t be shy: drop me an email at joel(at)getstation(dot)com and let’s chat! 🍺

Station over and out.