100,000 Users on Firebase Firestore

Introduction

I built an iOS, Android, and web app called Simple Secret Santa Generator using Firebase Firestore as the backend. At a userbase of 100,000 and with the desire to add complementary features and scale, I decided it is prudent to migrate the backend for this app from Firebase Firestore to one backed by Postgres. This article describes my experience with Firestore and explains my decision to migrate.

Firestore is Simple and Maintains Consistency

Even though I ultimately decided to migrate away from Firestore, I believe my initial decision to use Firestore was justified. Because I did not have to build a backend, Firestore enabled me to release quicker. Given that the vast majority of my projects fail, testing the market as quickly as possible is a virtue. As an additional benefit, it is simple to maintain consistency using Firestore because Firestore objects publish events when they update. These virtues come at a cost which is why I ended up migrating away from Firestore.

Breaking Good Coding Practices: Don’t Repeat Yourself

My app has three frontend applications. Consequently, I oftentimes had to repeat logic on the frontend that I normally would have to put on the backend. Repeating logic in multiple places caused difficulties in maintaining a consistent and reliable experience across three platforms. To minimize the repetition of logic, I utilized Firebase Functions in place of a traditional backend.

Difficulties with Firebase Functions

Many of my difficulties manifested while building out my backend functionality using Firebase Functions. While Firebase Functions are an amazing product, I met many challenges trying to maintain scalable software development practices.

The way the backend functioned was by utilizing Firestore triggers that invoke Firebase Functions. To illustrate how this works, I provide an example of sending out a secret Santa list. Each list has a `state` field. To send out a list, the front-end sets the `state` field to `SHOULD_SEND`. This triggers an update function. The update function sets the state to `SENDING` then performs the work. Once the work is completed, it sets the state to `SENT`.

Unfortunately, I ran into quality drawbacks by using this methodology. The first being issues with idempotence and atomicity. In the case of errors, it was oftentimes difficult to ascertain the correct state of the list because manipulating the underlying data is how the operation is initiated. This led to reliability issues in my app.

The other difficulties I experienced with Firebase Functions were testing and debugging. Testing was quite hard for me while I was building the various functions. The local testing did not work well with Firestore which necessitated that I deploy my functions for each test.

Finally, since the functions are tightly coupled to the underlying Firestore data, it became difficult to diagnose problems. There were many times where a user would give feedback about a problem that I could not reproduce because of uncertainty about the initial state. Because the functions are tied to the data, everything is tied together which makes the system difficult to maintain.

Now, I considered circumventing these problems by using Firebase HTTP Functions. Unfortunately, my app worked using the aforementioned reactive features of Firestore. The HTTP functions do not return a direct reference to the Firestore objects which breaks the reactive pattern my app was built with.

Responding to Customer Support Requests

Part of running an app that charges money is responding to customer support and feedback. In my case, I found using the Firestore user interface cumbersome. Operations that take seconds with SQL queries take minutes by navigating the Firestore user interface. This created a strain on my time as my user count increases

Summary

I enjoyed building with Firestore and it met the needs of my app as I proved it was viable in the market. However, as I plan to scale my app with new features and more users, it has become unsustainable and therefore I am rebuilding the backend with one backed by Postgres.