Introducing Firebase CTL

Firebase is like a Swiss knife for mobile developers. Most Android apps use one or the other feature of Firebase and likewise at Rapido we use Firebase for quite a few use cases. One such use case is remote configuration which helps control different business toggles and remotely configure the same. It can be changed dynamically expecting Firebase to do the heavy lifting of syncing it in almost real time which makes developers life easy. However, when there is a configuration and a web interface (Firebase console) to easily change one gets into this mode of changing configuration based on business needs manually which requires access control and is prone to human errors because of the lack of checks, validations and reviews.

Here comes firebase-ctl to the rescue.

One of the principles we hold dearly at Rapido is to keep everything automated, whether it’s configuration, code or deployments. We care about ensuring any change is duly reviewed before it hits production systems as even a small mistake in configuration change could potentially lead to disaster. As an example take a feature toggle, or a city identifier even if there is a single character missing in the configuration the application won’t behave the way it is intended to be.

One approach we took is to maintain our entire infrastructure as code including configuration. In this particular case we chose to keep all configurations checked in version control allowing us to leverage the following advantages:

• Makes it easy to review any changes with pull request model enforced
• It would make it easy for us to re-create environments with a click of a button with all configurations, infra etc intact.
• We have an audit of any changes to these configurations which becomes essential when you are dealing with production systems.
• Allows us to propagate the same configuration through different environments, testing environment to production environment enabling us to validate the changes.

Here are more reasons why we felt there is a need for a better way to manage this?

We use Firebase Remote configuration for making on-the-fly changes to our captain and customer apps. However, changing this normally would require one to have editor access to the Firebase project. Until very recently, we had to temporarily provide access to the concerned team member, and when the job was done, we used to revoke the access. However, we felt that this procedure had flaws. We could encounter many issues during the procedure. Some issues that we may face are: There could be mistakes in the privileges given to the user. The user may make some unintended changes, which gets passed in without peer review. Since we have fast release cycles, we may not be able to track all the changes to remote configs effectively. In case of a mistake, we might not be able to understand which change caused the issue, because comparing two versions of the configurations is cumbersome through the portal. We may forget to revoke the access after the activity is done.

So, we decided to create a tool that manages Firebase configurations.

How did we go about doing this ?

We had a good look into the Firebase REST API documentation, to understand its capabilities, especially how to control the full cycle of configuration management via APIs.x, which seemed to be straightforward. We decided to use Golang as a choice of programming stack to develop this tool, as we already used Go within Rapido (Check out Silent Assassin). However, we found that The Firebase Admin package in Go does not support remote configuration management. At the same time, we found a branch which did have some part of it implemented. This was not enough for us to use firebase-admin-go for our tool.

To write the unimplemented methods, we forked the firebase-admin-go package and temporarily renamed it. We implemented the functions and methods that let us create and modify the configurations. We followed this route because we did not want to re-implement the authentication and authorization flows provided by Google.

We currently only use conditions and parameters part of the remote-configuration API. So, we created a directory for conditions and one for parameters. Conditions are represented as an array, since they are ordered. So, conditions can be written as a JSON array, in a single file in the conditions directory. The parameters are represented as a map. Here, we are able to group the parameters by feature and write it to different files. The data will be read and could be processed to be applied to Firebase later on.

We used https://github.com/google/go-cmp package to compute the diff between source and remote config.

The tool, in the current state , has the following features:

Get: It gets the current remote configuration and dumps it to a directory. This gives you a starting point to drive your existing conditions and parameters through firebase-ctl

Diff: It gets the current remote configuration and compares with the one in the directory. The ones to be added to remote will be shown in Green and the ones to be removed would be shown in red. Makes it easier to identify the changes.

Validate: This command checks for any structural issues in the JSONs in the directory. However if the credentials are provided, it also makes a call to Firebase, which just validates the config, but does not apply it. This is a dry-run call, which does not change the remote value. Gives you faster feedback.

Apply: This command applies the current configuration available in the directory.

You want to give it a try ?

You can check out the tool from Github. Follow the instructions on the README.md to use the tool on your machine.

Running firebase-ctl would need a service account which has permissions to modify the remote-config parameters of the firebase project. The path to the service account should be exposed as an environment variable GOOGLE_APPLICATION_CREDENTIALS.

export GOOGLE_APPLICATION_CREDENTIALS=path/to/serviceaccount.json

Once this is applied, you’ll want to take a dump of the existing configurations to a local directory. You can run this following command to achieve this:

firebase-ctl get remote-config — output-dir test

If your credentials are configured correctly, a directory named test will be created in your current directory. The tool displays a message

Now, you may add some parameters and conditions to the conditions.json file, and create parameters in the parameters directory.

A sample condition which is to be added to conditions.json. The condition that gets appended to the file would be

A sample parameter you could add to parameters in the parameters directory by creating a new file.

Now, before applying any configuration changes, we may need to validate the structure of the configuration. To do this, we should execute the following command. If the credentials are provided with GOOGLE_APPLICATION_CREDENTIALS, an API call will be made to validate the structure. Else, just the JSON structure would be validated

firebase-ctl validate remote-config --input-dir test

To get the diff between the local and the remote values, we can execute

firebase-ctl diff remote-config --input-dir test

If there are additions, the values will be shown in green. If we intend to remove/modify values, the new value would be displayed in green and older ones(that gets deleted) would be displayed in red.

There you go, you have a command line tool which offers you a different lifecycle commands to manage remote configuration which can be hooked up to your Continuous Integration tool of your choice backed by source control repositories.

Feel free to provide comments or suggestions or even better contribute if you see this can be improved.

If you like the work we are doing and want to be part of this journey, we are hiring! Check out https://rapido.bike/Careers