Setting up a codelab or demo with Firestore data? Check out Firestore Export & Import.

You are planning to conduct a codelab that utilizes the Firestore database. The Firestore database and application that you have set up has 1000s of records. How do we set the stage for codelab or training participants to initialize and setup this data, so that they can focus on the rest of the application quickly.

This is exactly the requirement that I have faced multiple times in this year and I thought of documenting the process that has worked well for me. Its Firestore Export and Import feature.

Export and import data | Firestore | Firebase

The documentation is extensive and there are tons of features in there, but I will focus on the minimal feature that has allowed me to export the entire collection and restore it within seconds in another project. And that is exactly what your participants in a training or codelab session would need to do.

A bit of housekeeping first

Firestore Export and Import operations require a bunch of roles that span across operations that you can do with your Firestore databases and writing/reading of the data to export and which was exported respectively. I won’t rehash each of the permissions that are required, but suggest that you take a look at the Before you begin section of the Firestore Export and Import documentation.

Export and import data | Firestore | Firebase

My Firestore Database contents

As a sample, let us go with the scenario that I am in my Google Cloud Project and I have enabled Firebase in the project, have a sample collection in the (default) database in Firestore or it can be a database other than the (default) one too.

My sample collection is as follows:

I have a poses collection in the (default) database. There are 150+ records in this collection and each document in the collection is a rich metadata about a specific Yoga pose. You can see that one of the fields is an embedding field, which contains the vector values.

Exporting the collection

What I would like to do is to export the entire collection poses into a target destination first. Is this a single file or a folder that contains a bunch of files (data) along with its metadata? How does it all work?

Enter Firestore Export. It supports exporting your entire database or specific collections into a Google Cloud Storage bucket. While you can do the operations from the web-based Cloud console, we will take a look at gcloud operations, so that you can potentially automate and/or parameterize it for your requirements.

The process is going to be as follows:

1. Create a Google Cloud Storage bucket
2. Make this bucket public, so that anyone can access the bucket contents
3. Export the contents of the Firestore collection into this bucket

Before we jump into simply creating the bucket, a few caveats. You need to know the location of your Firestore database. This is required because the bucket will need to be created in the same region for Firestore export. That is one of the requirements.

The easiest way to understand your Firestore database location is to go to the Firestore page in the Cloud console and check out your database in the list. You will see Location as one of the columns. Note down that value. In my case it is asia-south1.

Let’s go now. The first step is to create the Google Cloud Storage bucket via the gsutil command shown below:

$ gsutil mb -l <<YOUR_BUCKET_LOCATION>> -b on gs://<YOUR_BUCKET_NAME>>

Once the bucket is created, we are going to make this bucket and its content public via the command shown below:

$ gsutil iam ch allUsers:objectViewer gs://<<YOUR_BUCKET_NAME>>

The final step is to export the Firestore database collection. In our case, the database is (default) and the collection name is poses. The command and its sample execution is shown below:

$ gcloud firestore export gs://<<YOUR_BUCKET_NAME>> --database="(default)" --collection-ids=poses

Waiting for [projects/gcp-experiments-349209/databases/(default)/operations/ASBmMTAzYmYzYTZmMjEtNjkzOC00YzU0LWVjNzMtOTJi
NzMyYmQkGnNlbmlsZXBpcAkKMxI] to finish...done.                                                                          
metadata:
  '@type': type.googleapis.com/google.firestore.admin.v1.ExportDocumentsMetadata
  collectionIds:
  - poses
  operationState: PROCESSING
  outputUriPrefix: gs://<<YOUR_BUCKET_NAME>>/2025-01-30T15:18:33_5234
  startTime: '2025-01-30T15:18:33.514196Z'
name: projects/<<YOUR_PROJECT_ID>>/databases/(default)/operations/ASBmMTAzYmYzYTZmMjEtNjkzOC00YzU0LWVjNzMtOTJiNzMyYmQkGnNlbmlsZXBpcAkKMxI

This would be a good time to take a look at the Exported content, which is a LevelDB database. A snapshot of my exported bucket and its contents are shown below:

We need to make a note of the path of this folder at the top, since we will need to provide that in the next section when we do an import. So for example, the path is gs://my-fs-export-bucket/2025–01–30T15:18:33_5234/ in my case.

At a generic level, it will be of the form:

gs://<<YOUR_BUCKET_NAME>>/<<EXPORTED_FOLDER_NAME>>

Importing the collection

Now that we have our database and collection available in a publicly accessible Google Cloud Storage bucket, we shift gears to another project and its Firebase Database into which we want to import the data.

So assuming that you are in this new project, the steps will be as follows:

1. Create a bucket in this project.
2. Copy the database export that we have prepared into this bucket, before we can import it into the Firestore database.
3. Import the contents of the bucket into the Firestore database.

Lets go. The first step is to create a bucket (for e.g. gs://my-firestore-db-import-bucket), so let’s create that first.

$ gsutil mb -l us-central1 gs://my-fs-import-bucket

Download the contents of the previously exported bucket into this bucket. We are using the gsutil cp command recursively to take all the contents from the bucket/folder exported in the previous section into this new one that we created in the first step.

A sample execution is shown below:

gsutil cp -r gs://my-fs-export-bucket/2025-01-30T15:18:33_5234  gs://my-fs-import-bucket                                                   

Copying gs://my-fs-export-bucket/2025-01-30T15:18:33_5234/2025-01-30T15:18:33_5234.overall_export_metadata [Content-Type=application/octet-stream]...
Copying gs://my-fs-export-bucket/2025-01-30T15:18:33_5234/all_namespaces/kind_poses/all_namespaces_kind_poses.export_metadata [Content-Type=application/octet-stream]...
Copying gs://my-fs-export-bucket/2025-01-30T15:18:33_5234/all_namespaces/kind_poses/output-0 [Content-Type=application/octet-stream]...
| [3 files][  3.6 MiB/  3.6 MiB]                                                
Operation completed over 3 objects/3.6 MiB.  

Finally, we import this new bucket/folder into our firestore database via the import command given below along with its sample execution:

gcloud firestore import --database="yoga-database" gs://my-fs-import-bucket/2025-01-30T15:18:33_5234
Waiting for [projects/gcp-experiments-349209/databases/yoga-database/operations/AiAyYWI4NTcxMDllMjQtYzBmYS0xMzI0LTc2OTktZGFjMGVhNmYkGnNlbmlsZXBpcAkKMxI] to finish...done.                        
metadata:
  '@type': type.googleapis.com/google.firestore.admin.v1.ImportDocumentsMetadata
  inputUriPrefix: gs://my-fs-import-bucket/2025-01-30T15:18:33_5234
  operationState: PROCESSING
  startTime: '2025-01-30T15:43:30.245200Z'
name: projects/<<GCP_PROJECT_ID>>/databases/yoga-database/operations/AiAyYWI4NTcxMDllMjQtYzBmYS0xMzI0LTc2OTktZGFjMGVhNmYkGnNlbmlsZXBpcAkKMxI

That’s it. We can check in the yoga-database if the collection has been created and there it is.

This concludes the tutorial on Firestore Export and Import, which is a handy way to not just take backups at some point in time, but also a way to create the collections at scale in learning environments.