The Four Types of Feature Flags
Last year, I held a presentation at DevFest 2022 in which I talked about how, by using feature flags, we managed to increase the speed at which we delivered our mobile updates and decreased the time it takes for us to restore services if something doesn’t work as expected during the release.
The primary drive for implementing feature flags was to reduce deployment anxiety while offering a sense of psychological safety for the team. This meant we needed to create a system where features could be rolled out incrementally, validated with a small number of users, and disabled if any issues arose.
Feature flags provided a safety net by reducing the risk of negatively affecting our users and also they helped stimulate experimentation, increase speed and agility, and enhance our feedback loops, essentially it paved the way for us to achieve higher performance.
From our experience with feature flags, we separated them into four types:
- Access
The classic feature flags can hide features or parts of the app from your users. - Release
Allow us to experiment and use trunk-based development practices. - Experiments
These are improvements that we are not sure yet how they will impact the overall performance of the app, so we want to enable or disable them. - OPS
Graceful degradation of the system sometimes allows us to maintain the overall stability of the system by closing down small parts of the app.
We will go into detail about each feature flag in the hope that by understanding the unique characteristics and purposes of each one, we can better appreciate the versatility and usefulness of feature flags in modern software development.
Access
These are the classic feature flags, sometimes called toggle flags, they are often used for canary releases or controlling the access to certain parts of a system.
Based on certain user characteristics (such as OS, subscription status, location, or previous actions in the app) we will display particular features to them in the hope of gathering feedback and observing their behavior as well as the app’s stability and system performance.
For example, when Pago launched the vignette & bridge toll products, we first gave access to people who already had a car added to the app from the insurance section.
Once enough data is gathered or when we’ve decided that the feature is stable enough, we roll out the feature to all of our users. The opposite can be true as well, if we detect that the product would cause issues for our users, we’d roll back the feature and come up with a fix for the next release.
Release
Release feature flags enable agility and experimentation; we can launch unfinished features in production to gather real-world insights on the health, stability, and performance of our as-yet-unreleased features.
We had this approach with our Travel & Property insurance releases in Pago and in 2022 it proved helpful. In February, four months before the official release, our code was in production, hidden by feature flags and collecting data.
We managed to discover several issues in production, correct them, and even make some improvements:
- An improperly executed database migration
- An API endpoint that exceeded the expected response time under heavy load
- An API that triggered too frequently for specific data
Although we had to address each issue with each release that we did, I’ll always prefer fixing 100 small issues at a steady pace over four months instead of dealing with one huge fire when we release the new feature.
Experiments
Experiment feature flags enable our developers to be innovative and take calculated risks. For instance, if we want to evaluate the efficacy of a new database (say a migration from SQL to NoSQL), no matter how much developer testing and benchmarks you’re going to do in development, it’s not going to be the same thing as getting accurate user data.
To experiment with this, we migrated a limited number of tables to NoSQL, and using a feature flag we directed some queries to be run on the NoSQL database while others on the SQL one. Finally, we crunched up the numbers, got the results, and decided on our best approach.
The central idea behind this type of feature flag is to make developers understand that experiments are not failures, but only learning opportunities. This mindset is crucial, especially because at any time, we can just roll back the change.
OPS
OPS feature flags allow for graceful degradation of the system, leading to a better overall performance of the app. This is usually done at times when you need to focus your resources and preserve the overall stability of your system to the detriment of certain functionalities or worse UX.
For example, during the pandemic, a surge of new users in the app led to increased pressure on our support department. The users were using the “suggest supplier” button instead of the search function, even though the desired supplier was already in the app.
To address this, we made the button less prominent based on our capacity. If we had the capacity, we would display it on the front page without the need for a search (as shown in Image 1). If not, it was only displayed after a search that produced no results (Images 2 and 3 below).
Learnings
Feature flags have become a crucial tool in the way we develop and deploy our code. They allow us to slowly roll out new features, test them with a limited subset of users, and gather feedback before a full release.
Looking at the development landscape the need for faster release cycles and the trend to be more agile, feature flags should be considered a must-have tool for any team that wants to optimize its processes and deliver high-quality software.
For us, at Pago, we have 4 big accomplishments thanks to feature flags:
- We removed the fear of creating a release
- Decreased the lead time for change
- Increased the deployment frequency
- Improved the time to restore service
If you liked this article, you might want to read about the challenges we’ve faced when working with feature flags and how we overcame them.
Oh and by the way, we’re always looking for new people to join us at Pago. Check out the available jobs here. In case you don’t see a job opening for your role, but you think you might be a good match, drop us an email.
