Effective Cloud Firestore — Part 1
The pros and cons of Cloud Firestore
This is the series of articles describing how to use Firestore effectively. As a part 1, I’ll tell you the pros and cons of Cloud Firestore.
(日本語版はこちらです: Cloud Firestoreの勘所 パート1 — Cloud Firestoreの概要)
Cloud Firestore is regarded as one of the service of Firebase or one of the services of GCP(Google Cloud Platform), and I’ll treat Cloud Firestore as the former. This also applies when treating other services such as Cloud Functions. In short, Firebase is the wrapper service of GCP especially for client applications, and it is more convenient than using GCP directly in general, so I’ll recommend you to use the services via Firebase if possible.
There are documents for Firebase and GCP and they resemble each other, and so it is enough to read the former basically 👍
In the first place, if you think “what’s Cloud Firestore”, refer to this article (mainly first half).
(Sorry, this is written in Japanese 🇯🇵 🙇)
Benefits of using Firestore
First, I’ll tell you the benefits of using Firestore.
Serverless (No need to manage servers)
No need to manage servers by using Firestore and it is common to all Firebase services, so you’ll get these benefits:
- Cut the cost of managing servers
- Obtain high availability
But, Firestore is still beta now, so you should pay attention to these restrictions:
- Some beta limits
- Out of SLA(Service Level Agreement) and Deprecation Policy
Indeed, you may concern about “out of SLA”, but it works very well these days in my experience. Until 2017, I felt some buggy behaviors and latency, but I cannot feel such strange behaviors recently 👍
You can check status information on Firebase services, and there are no incidents last week as you can see.
Although SLA is not applicable for Firestore still, I think it can be used for production services. But this depends on personal feelings and the service policy, so it is case by case whether you can use Firestore for it. I hope that the official version of Firestore will be released soon 🐶
Extremely high scaling performance
The official document refers Firestore as this:
Cloud Firestore brings you the best of Google Cloud Platform’s powerful infrastructure: automatic multi-region data replication, strong consistency guarantees, atomic batch operations, and real transaction support. We’ve designed Cloud Firestore to handle the toughest database workloads from the world’s biggest apps.
You can develop the service with extremely high scaling performance easily by using Firestore.
High-performance data synchronization can be easily realized only with client application code
From now on, I’ll explain Firestore to you from the point of view of actual use. Traditional Web API — client application system can be easily realized only with client application code thanks to Firestore (This also applies to Firebase Realtime Database).
If you write the client application code like accessing the local database, the local data will synchronize to the cloud database(actually Google Cloud Datastore). And, the Firestore SDK handle offline situations very well.
- If data accessed, returns the cached data immediately
- If data updated, it stores data on local only temporarily and it will be synchronize immediately after connected to the network.
It was difficult to implement traditional Web API — client application data synchronization by oneself, and it is more difficult to care about scaling performance. In the small number of people development so far, it was common to compromise something, but it can be realized to develop high-quality web services by small team in a short-term by using Firestore.
### Custom cloud logic can run at some point
Firestore doesn’t restrict you only to writing client applications code, but you can write cloud logic if you want by combined with Cloud Functions. Firestore has these flexibilities:
- Firestore Trigger: Run cloud logic from data added/updated/delete trigger
- HTTPS Trigger: Call Cloud Functions from a client application and the functions update Firestore data from there
Disadvantages of Firestore
I mentioned advantages of Firestore so far, but Firestore has some disadvantages of course. They are mainly caused by Firestore being a NoSQL database.
Difficult to run the complex query / do aggregation processing
As a prerequisite, Firestore can run some queries, and this is one of the improved points from Realtime Database.
Perform Simple and Compound Queries in Cloud Firestore | Firebase
Create Messaging Experiments in the Firebase Console
These queries are supported on Firestore:
- Simple filter/sort by a single field
- Simple filter/sort by multiple fields
By default, all single fields are indexed so that the former can be queried. But if you want to query by multiple fields, it is necessary to create the index in advance depending on the fields combination.
Manage Indexes in Cloud Firestore | Firebase
Cloud Firestore requires an index for every query, to ensure the best performance. All document fields are…
If you execute the query with multiple fields, it fails with this error at runtime, and it demands you to create the index.
Error Domain=FIRFirestoreErrorDomain Code=9 “The query requires an index. You can create it here: https://console.firebase.google.com/project/PROJECT_ID/database/firestore/indexes?create_index=EghtZXNzYWdlcxoNCgljcmVhdGVkQXQQAxoQCgxzZW5kZXJVc2VySWQQAxoMCghfX25hbWVfXxAD"
If you click the link, it opens the dialog to create the missing index.
By the way, you can create the index on the web console directly, but I recommend you to use Firebase CLI for the production development. Firebase CLI is useful for versioning and accurate operations for multiple projects environments.
The index on the figure above is same as this code:
You can deploy it with this command:
firebase deploy --only firestore:indexes
You might think that it is annoying to create indexes every time multiple fields are used, but this guarantees that all of the queries are fast thanks to the indexes. You cannot run a slow query with this restriction and it is one of the good points of Firestore 👍
Simple means that Firestore query has some restrictions, and Firestore is inferior to RDB(Relational Database) at this point. You can imagine the restrictions by reading “Query limitations” on the document, which notes 5 restrictions. Personally, I’m bothered by not being able to run queries with a
!= clause. The document says that this can be solved by combining two queries by using
> , but this cannot filter
NOT NULL documents. In this case, you should adjust the data structure, or get all documents and filter out the documents which have
And, Firestore doesn’t even have concepts like
JOIN or aggregation operations such as
GROUP BY which SQL has, so you should adjust your mind to fit to Firestore as these:
- When you use RDB, it is better to normalize the structure and get the results you want by querying.
- When you use Firestore, which is NoSQL database, it is better to design the data structure so that you can get documents with a simple query(no filter conditions is better).
- If you cannot realize the specification only by devising the data structure, you should combine other services (There is a Full Text Seach solution on the official document).
- Data analysis with Firestore is almost impossible, and it is necessary to preprocess by using data store for it such as BigQuery (Or you can analyze data contained in Firestore by full scan data and writing the code for it, but it is not recommended).
Data backup managed service for Firestore is not provided yet
There are some questions on Stack Overflow etc. about data backup service for Firestore, and it is not provided yet as answered there.
Firestore new database — How do I backup
Join Stack Overflow to learn, share knowledge, and build your career. Does the google firestore database service…
But, node-firestore-backup, which is a backup script for Firestore exists. You can backup Firestore data if you execute this script periodically by using Cron Job etc. Of course, you can write your own backup script.
node-firestore-backup — Google Firebase Firestore backup tool
That is, you should choice of these for Firestore data backup:
- Create a mechanism to run node-firestore-backup etc. periodically
- Or give up backup…
There is an official backup/restore service for Firebase Realtime Database and this is good service. You can backup Realtime Database on Google Cloud Storage(GCS) every day with a simple settings.
Automated Backups | Firebase Realtime Database | Firebase
We offer an easy to use configuration switch that enables a default 30 day object lifecycle policy for your Google…
I expect that similar service is offered officially when the beta ended 🧐
To wrap the characteristics of Firestore up, I can say as follows:
- It is important to understand Firestore properly and think about data structure that fits to your application specification.
- It is possible to query/sort for Firestore data, but you may have to use other services together for complicated problems.
- If you’d like to analyze Firestore data, you should prepare other data store for it such as BigQuery, and it takes some time and effort.
- Notice that Firestore is still in beta (SLA not applicable and no managed backup service, etc.)
The disadvantage of Firestore is mainly due to being a NoSQL database, and it should be regarded as a tradeoff with extremely high scaling performance. I feel appreciate even simple queries can be done 👍
I’ll explain how to use Firestore in practical by thinking data structures on the article part 2.
(Only available Japanese version article still now 🇯🇵 🙇)