Feature flags & visual testing for complete launch confidence
Feature flags are all about helping product teams deliver faster by managing control and access. They’re not new, but they’re on fire, and teams are using them for everything — internal testing, beta testing, A/B testing, geo personalization, advanced segmentation, so on and so forth.
By separating deployments from feature releases, developers can safely test in production, get valuable feedback, and release new features continuously with less risk.
At Percy, we use LaunchDarkly 🚀 to manage our flags and segments to help us feel more confident in developing and delivering new features by incrementally rolling out and thoroughly testing our code with real users in production, including our own team internally.
As more flags get thrown into the mix, however, the overhead required to manage them can be significant — it exponentially introduces new states and combinations you need to be aware of. The ever-expanding permutations of states that pages and components can be in makes testing more and more challenging—both for manual and automated testing. It can be pretty difficult to know with absolute certainty what’s being seen across all those combos, and how they might affect one another.
To help teams handle that level of complexity, we’ve found that combining feature flags with Percy’s own visual testing can provide a unique and easy way to gain confidence in testing feature flags and variations.
Feature flags for testing in production
Teams are already using feature flags for testing purposes — turning on flags for QA teams to manually test in production. But the pervading challenges of manual QA still exist. It’s slow, expensive, error-prone, and now you’ve introduced the difficulty of enabling combos of flags for QA testers, and knowing and remembering exactly how each variation and combo should look.
Each flag you add exponentially increases the combinations of variations, and exponentially increases the cost of manual QA.
With visual testing, you can automate that manual work. Instead of having to manually enable each combination of flags to review the screens manually, visual testing gives you the ability to continuously see and review the combinations of flags and make sure they look exactly as you’d expect.
Using feature flags at Percy, with Percy
We recently launched an exciting new feature affecting both our front and backend — automatic diff matching. Once the backend functionality shipped and design and development on the frontend started taking form, we added a feature flag that we could enable for the team.
We wanted to use the functionality ourselves and iterate on design throughout that process. During that time, we needed to keep our “off” variant stable and unchanged while iterating on our feature flagged variant. With visual testing, we were able to continuously monitor all branches, eliminating the risk of visual regressions.
At the same time, we could iterate upon and review all UI changes on our “on” variant. Once we were happy with all design iterations, we turned the flag on for a group of beta testers while maintaining visual reviews for both variants.
With visual testing integrated into our code review process, it was easy for us to review visual changes on every commit, even for a feature that was only enabled internally while under development.
In the end, we gained better test coverage since we had tests for the feature flag off, as well as new tests for the feature flag being on. Because the feature will eventually become a permanent part of our code base that is enabled for everyone, we can simply remove the variant of the tests with the feature flag off when we’re ready.
This process has allowed us to simultaneously test all of our variants, giving us constant insight into what’s changed, and confidence that all versions of our app look exactly as they should.
It’s a great feeling to be able to ship iterative changes to a new feature, and know that you aren’t breaking your existing app.
This is a true case of 1 + 1 = 3. Both visual testing and feature flags on their own reduce deployment friction and risk. Together, there’s an opportunity to take agile development and deployment to another level — achieving a level of confidence most of us as developers haven’t ever really had.
Whether you’re using flags for feature testing and release management, or long term segmentation and personalization, adding visual testing alongside feature flags is a powerful combination that can help teams feel more confident and productive while iterating quickly on new features. It has certainly helped us at Percy, and we continue to experiment with new ways to combine visual testing and feature flags in our own development.
Are you heading to Trajectory Conference on April 9th? We’re thrilled to be sponsoring — stop by our booth to learn more about visual testing and tell us about your feature flag workflows.
Percy is an all-in-one visual testing solution that gives your team confidence in every visual change before it’s shipped. Learn more at https://percy.io. 🚀
