How Do You Eat An Elephant?

The personalization squad at the hot new FinTech company Bankake spent 3 months building a feature that the CTO hailed as “one of the greatest engineering achievements [he’s] seen in the past year.” They monitor the feature’s performance over the next few weeks only to discover it’s a total flop. So what happened? They realized a little too late that sometimes the most impressive work isn’t the most necessary work.

Here at Stash, we try to work in a more iterative, logical process. I’d like to propose a simple framework to help you maintain this discipline whether you’re at a start-up, a scale-up (apparently this is a thing now), or a mature company:

1. Identify a need
2. Make a hypothesis
3. Build an MVP
4. Validate the hypothesis
5. Repeat

1. Identify a need

This is often the most skipped step in the process. With so many new technologies and techniques, it’s easy to choose work for no other reason than desire. But how desirable is work that ends up trashed after failing to prove its value? If you’re approaching a new product, it’s important to first validate that the product is necessary. If you’re iterating on a product, it’s important to use learnings from the previous iteration to validate the changes. Be honest with yourself: Is this worth doing? This step can save a ton of time and generally doesn’t involve too much effort. Here are a few tricks we’ve used in the past:

• Create a waitlist to gauge interest in a yet-to-be-built retirement product
• Analyze drop-off rates to identify weaknesses in an on-boarding flow
• Send targeted messages to gauge if users are interested in news articles related to their investments

For the sake of this post, let’s imagine you’re inventing chocolate cake. Before experimenting with recipes, you may want to validate that people are gluttonous, that people like chocolate, and that they like vanilla cake but want an alternative.

2. Make a hypothesis

A single iteration should do a single thing. What is it? Before you begin work, identify what you expect the outcome to be and formalize what metrics you’ll use to validate this hypothesis.

• Hypothesis: Customers want chocolate cake.
• Evaluation Metrics: (1) sales, (2) customer satisfaction

3. Build an MVP

Build a Minimum Viable Product with an emphasis on minimum. In other words, only build the features that are absolutely necessary for you to get the product to customers and start learning. You don’t want to spend weeks formulating batter, frosting, Oreo crunchies, and raspberry sauce just to find out people don’t like your cake. Is there no market for chocolate cake? Does raspberry sauce have no business on top of chocolate cake (it doesn’t)? Now you’re stuck untangling the feedback trying to figure out what to do next. If you’re trying to validate that people like chocolate cake, just make the cake. No frosting. No crunchies. Definitely no raspberry sauce. This is what we call feature creep, and feature creep only makes things messy. Again, a single iteration should do a single thing.

4. Validate the hypothesis

Remember the metrics you formalized in Step 2? Check them. There are considerations you need to take to make sure the metrics are significant, but since they are out of the scope of this post let’s just assume you’ve let the feature properly simmer. It’s tempting to try to rationalize your work and say, “Well we expected sales and customer satisfaction to go up and they didn’t — but look! This other irrelevant metric seemed to improve a bit!” There’s nothing wrong with shooting and missing. Stick to the core metrics, and if they validate your hypothesis, then continue down that path. If they don’t, take a step back and use the results to help formulate your next hypothesis.

5. Repeat

Given your results, decide what to do next. Assume sales and customer satisfaction both improved with your new chocolate cake. Now, you can start to add the frosting and crunchies, but only one at a time! Frosting could be a 5% lift, crunchies could be a 6% lift, but together they could be a 2% lift. A disciplined, iterative process should produce this information without you ever needing to backtrack or guess.

Wrapping it all up

Obviously there are some heavy projects, but generally if you’re working for several weeks without any new learnings there’s a chance you’re not breaking your projects down enough. Give this a try and let me know what does or doesn’t work for you. If this type of process appeals to you, check out open jobs at Stash here.