Firebase Schema Evolution

Schema evolution is a natural part of your application lifecycle. Firebase is my go-to backend for web and mobile projects and I rely on two technologies when refactoring the data model of an app:

• Bolt as the single source of truth for both the backend (rules & cloud functions) and the client (Typescript). This prevents the backend and the client from evolving separately: Firebase cannot have a type definition that is different from the client and vice versa.
• JSONiq to quickly develop migration queries that can be executed on large datasets. Using JSONiq, it is possible to express complex data transformations with only few lines of code.

Single Source of Truth with Bolt

Bolt is modeling language for Firebase. By default, the Bolt compiler outputs Firebase rules that can be automatically deployed to the database. This feature alone is already a life saver since Firebase rules can be incredibly cumbersome to write manually. On top of that, there is also a compiler to Typescript available that allows you keep your data model in sync on the client. The Typescript definition can also be used to implement Cloud Functions. I haven’t tried yet to output type definitions for flow but it should be fairly easy to do as well. Other asset types such as documentation or customized code artifacts could be generated as well.

Let’s look at the following Bolt definition.

This file can be used to deploy the database security rules to firebase and to generate the data model on the client. To execute the first task, you can use the firebase-bolt command-line tool and curl.

The firebase-bolt-transpiler command can be used to generate typescript command. Since src/Model.ts is a generated file, you can add it to your .gitignore.

The generated file can be used when fetching data as below.

Now if you rename a field in the bolt file without refactoring the code properly, it won’t compile.

Now that you are certain to have the Firebase rules and the client in sync, the last step is to migrate the legacy data to the new schema if necessary.

Data Migration with JSONiq

The JSONiq query language enables you to easily express data transformations that otherwise would be a nightmare to write in Javascript. Personally, I like use the Zorba docker image to execute JSONiq code.

The following JSONiq query fetches a document from firebase and applies a simple transformation to it before sending it back to the server.

To execute this query, you need to set a proper Firebase token, which you can do with the following command.

JSONiq provides capabilities to interact with the Firebase REST API easily. The following is the implementation of fb:get(), fb:put(), and fb:delete().

In this last example, we remove data that has been duplicated from multiple locations since it is a common pattern in Firebase.

Et voilà

I hope this article was helpful to you, and I look forward to receiving your feedback.