Mastering Feature Flags: The Basics

Martin Chaov
DraftKings Engineering
6 min readNov 18, 2023

This article is the first one in a series, more advanced topics are covered in the subsequent issues:

What Are Feature Flags?

Feature flags, also known as feature toggles, have become a cornerstone in modern software development, enabling teams to deliver features incrementally, test in production, and respond quickly to market changes. This guide covers everything one needs to know about feature flags, from best practices to resilience and performance considerations. This article is not a tutorial on using a specific third-party service to handle feature flags but a guide on what is required to run a successful operation depending on feature flags.

Feature flags are conditional code statements that allow developers to toggle features on or off without redeploying the application. They facilitate continuous delivery, enabling code to be merged into the main codebase and deployed while keeping new or incomplete features hidden from the end-users.

A feature flag can be configured to serve different values based on a user segment.

Feature Flags != Remote Configuration

While feature flags and remote configurations control system behavior, they serve different purposes. Feature flags control access to specific features, whereas remote configurations set operational parameters. Feature flags are naturally binary, on or off, while remote configurations can manage more complex settings.

Feature flags (a.k.a. feature toggles) are a way for engineering teams to instrument an application to turn features on or off under specific conditions.

A feature flag is a fine-grained business behavior control mechanism. While achieving the same with RC is somewhat possible, there are brutal fundamental differences.

With remote configuration, an application typically retrieves a block of settings from its back-end or configuration service. Then, the application configures its settings and features based on it.

  • Configurations are settings that control the behavior of a system. They are typically used to set the system's parameters or operating options.
  • Configurations are often set at startup and do not change while the system is running unless the system is restarted or specifically designed to allow dynamic configuration changes.
  • Testing in multiple environments

Why Use Feature Flags?

  • Operational Risk Management: Safely test new features in a live environment.
  • Product Experience Optimization: Conduct A/B tests to improve user experience.
  • Revenue/Engagement Tests: Measure the impact of new features on key performance indicators.
  • Decouple deployment and release: Engineers can deploy new features but not release them to the end users.

When to use feature flags

  1. Gradual feature rollout — roll out new features to a small percentage of users while gradually ramping up the % while monitoring the overall system.
  2. A/B Testing and Experimentation — show feature variations to determine the impact on various KPIs.
  3. Canary releases — releasing features to a small segment of users: beta testers, early adopters, etc.
  4. Trunk-based development and CD — used to turn off incomplete features but still merge into the codebase and deploy to production.
  5. Operational switches — application level controls such as log level, rate limiting calls to BE, etc.
  6. Permissions — use flags to control access to features based on user segmentation.

When not to use feature flags

  1. Control of sensitive data — flags should not be used to control access to sensitive data or critical system functionality.
  2. Managing uncertainty — feature flags are not a way to avoid deciding "if" to release a feature.
  3. Permanent features — some features will always be enabled and won't need to be hidden behind a flag: login, deposit, withdrawal, etc.
  4. Instead of configuration — feature flags should not substitute remote configuration.

Pitfalls of Feature Flagging

Increased code complexity

Implementing a new Boolean flag adds two new code paths in an application, but it could also mean two different versions of a service (or entirely different back-end) running side-by-side. Multiple flags related to the same feature create a permutation of possible code paths. Example 3 flags with two possible states is 2³, which gives eight different code paths. Only a few of these eight permutations could be related to viable business flows, and the rest should not be possible — for which the application must ensure they are not.

Increased testing complexity

Increased cyclomatic complexity in the code comes with overhead for quality assurance. 3 Boolean flags can be combined in 8 ways; 10 gives 1024 possible combinations. This is added to the required features so a feature flagging system can manage them. This becomes exponentially more complex when flags get more than two possible values each.

Overuse

Some of the use cases for feature flags are prone to overuse. It is easy to push code to the stream behind a flag. However, is that code going to be ever released? Ideally — yes. However, with the dynamic nature of business niches and priority shifts, this code could become a "zombie" of a feature that will never be released. Still, it got merged into the codebase and deployed to production. It is an unfinished feature, never to be completed, hidden behind a flag that will never be turned on.

A problem with putting every feature behind a flag is that eventually, it is not only the different combinations of flags that should be tested but also compatibility between what features are enabled.

In addition, this could also lead to increased package size for native client applications.

Glossary

  1. Feature Flags: Conditional code statements that allow developers to toggle features on or off without redeploying the application.
  2. Remote Configuration: A block of settings retrieved from a back-end or configuration service that controls system behavior.
  3. Phased Rollouts: The practice of releasing new features to a subset of users, gradually increasing the percentage of users who see the quality.
  4. Kill Switch: A feature flag that can immediately disable a feature in case of an issue.
  5. Chain of Dependencies: The interrelationship between multiple feature flags, where one flag's state may depend on another's state.
  6. Operational Risk Management: Using feature flags to test new features in a live environment safely.
  7. Product Experience Optimization: Using feature flags for A/B testing to improve user experience.
  8. Revenue/Engagement Tests: Experiments using feature flags to measure the impact of new features on key performance indicators (KPIs).
  9. Canary Releases: The practice of releasing features to a small segment of users, such as beta testers or early adopters.
  10. Trunk-Based Development: A practice where all developers work in a single branch, using feature flags to toggle incomplete features.
  11. Operational Switches: Feature flags for application-level controls like log levels or rate-limiting back-end calls.
  12. Cyclomatic Complexity: A measure of the number of linearly independent paths through a program's source code, increased using feature flags.
  13. Zombie Feature: A feature hidden behind a flag and never released, leading to codebase clutter.
  14. Package Size: The total size of the application, which can increase due to the overuse of feature flags in native client applications.

Summary

Feature flags are potent tools for agile development, balancing speed and safety. Effective use requires a disciplined approach to testing, documentation, and ongoing lifecycle management. Understanding and implementing these best practices can maximize the benefits while mitigating risks.

This guide aims to be the go-to document for mastering feature flags, covering everything from basic concepts to advanced best practices. A standardized approach can ensure consistent, efficient, and safe use of feature flags across all projects.

This article is part of a series; if you enjoyed it, you should check out some of the other parts as well:

Want to learn more about DraftKings' global Engineering team and culture? Check out our Engineer Spotlights and current openings!

--

--

Martin Chaov
DraftKings Engineering

15+ years as a software architect, currently Lead Software Architect at DraftKings, specializing in large-scale systems and award-winning iGaming software.