Phoenix — Tinder’s Testing Platform — Part II
Written by:
Connor Wybranowski, Software Engineer | Siggi Jonsson, Staff Software Engineer | Keith McKnight, Staff Software Engineer | Juzheng Li, Engineering Manager
In this section, we’ll discuss how we address our specific needs around remote configuration and decoupling experience delivery from app releases. We suggest reading our first article to gain a holistic overview of our experimentation platform before continuing here. Once you’re ready, let’s dive in!
A lot of work can go into creating a new feature for Tinder. Product Managers and Designers need to figure out what the feature should do and how it should look and feel. Engineers need to write the code that makes it work. Quality Assurance needs to test things to make sure that everything is working properly and that the feature doesn’t create trouble in other parts of the app. Binaries need to be compiled and sent to app stores for approval.
Even after all of that, there can still be surprises. Maybe we noticed a bug that we didn’t catch during QA while rolling out the new app version. Maybe a sudden flash of inspiration required tweaking some details to make the experience that much better. Maybe everything is good to go, but we want to give the feature a splashy launch at a specific time. Any change can require going through the entire process again, and that can really throw things off.
The ability to decouple experience delivery from app releases on the mobile clients is a critical part of our workflow at Tinder. After all, there are a number of reasons why feature releases and app releases don’t always align on a consistent schedule.
Remote Configuration
Remote configuration is a straightforward concept with powerful implications. Rather than baking parameters and feature flags directly into the binary, we can allow the app to request the configuration from our servers. It enables the configuration of the Tinder app from outside the app itself, allowing us to change how the app behaves outside of the release cycle. We can even automate it by scheduling configuration changes.
Although this approach can be used on any type of client independent of the release cadence, remote configuration really shines on mobile apps, where we cannot simply revert and redeploy if something goes wrong.
Levers
We call the fundamental unit of our remote configuration system a “Lever.” Each Lever consists of an identifier, a type (Boolean, String, or Number), and a value. This limited complexity is flexible enough to describe fairly robust configurations while still being straightforward to implement consistently across multiple platforms.
Designing Configuration with Levers
The configuration described by Levers encourages granular feature control that is platform-agnostic and product-focused. We can reason about parts of the feature that might make good candidates for an experiment before writing any code, which can in turn help us to make better design and architecture decisions.
One common pattern is to use separate Levers to control the primary gating of the feature itself as well as any parameters of the feature. For example, a new flow might use String Levers for the member-facing copy and button color, another Number Lever to control the delay before a notification pops up, while still maintaining a Boolean Lever that acts as the primary control for whether the member sees the new feature at all.
The API response might include something like this:
Managing Levers with Config Service
The backend is responsible for constructing the payload of Levers when the client requests it. On its own, this payload is purely a delivery mechanism for the remote configuration. On the client-side, we created a “Config Service” on each platform, which is an SDK to process, persist, and vend new values from our Phoenix system. While each Config Service implementation addresses concerns that may be specific to a certain platform, we relied on cross-platform collaboration to achieve feature parity across the iOS, Android, and Web clients.
One “gotcha” about decoupling the configuration from the client is that the backend might be out of sync with the client. Fortunately, the nature of integrating the Lever into the feature leads us to make some assertions. It wouldn’t make sense for the client to want to interpret the same Lever as two different types, therefore the client must inherently know the type of Lever it is using. This also implies that the client knows all the Levers that it might read.
We created a manifest to declare all the Levers and their corresponding types, which allows us to validate the values that the backend sends and ignore anything that would violate the type of Lever the client expects. We also declare fallback values in case the server doesn’t send a particular Lever, such as what might happen on an older client after we’ve deprecated a Lever on the backend.
To add more flexibility, we defined different locking policies (unlocked, session-locked, etc), which regulate when a Lever would take effect after its receipt from backend.
Experimentation
So far, we haven’t said much about experimentation, so let’s put this into context. Just like remote configuration provides a mechanism for adjusting the app experience, experiments provide a mechanism for adjusting the configuration. Experiments allow us to propagate remote configuration to clients based on targeting criteria, quickly determine if new products are working as intended for our members and measure their impact.
The Config Service itself doesn’t really have a concept of an “experiment.” It doesn’t make any assumptions about why the configuration is a certain way; the configuration just is. Each Lever comes with an identifier that tells the backend whether the member’s experience is associated with an experiment. The identifier has no particular meaning to the client, but it is included with any key metrics that the feature tracks. This helps us understand the impact of new features powered by Levers while maintaining a separation of concerns.
Metrics
In the past, we’ve found that while ensuring that our experiment data is accurate and valid is of critical importance, it’s really easy to make mistakes when instrumenting analytics from scratch. We thought that developers should get robust reporting for free by using Levers and Config Service, so we made tracking new values on Lever reads a core feature. This improves the quality of our analytics and reduces the burden of each team to report experiment variants and assignment.
In our next blog post, we will go into greater detail about how we collect metrics to produce valuable and actionable insights based on experimentation.
So, How Did We Do?
Config Service and Levers came out of a need for a robust experimentation platform and it ended up offering us more than that. Decoupling feature configuration from the app release cycle gives us an extra measure of safety. It’s neither trivial nor fast to roll back an app release after there is already a problem. However, rolling back a configuration change can be done in a matter of seconds.
It allows us to take a more nimble, iterative approach to feature development. We want to avoid the case where new work lands late in the release cycle without sufficient time to identify potential bugs. We can land new work on primary development branches early and harden the implementation without fear of shipping an incomplete feature into production.
We love what we’ve been able to accomplish with this, and we hope you will too.
Interested in joining Tinder? Click to explore open opportunities.
