
But first be so convinced that your feature is a great idea that when the results come in and they are negative, your first instinct is to figure out what’s wrong with the test. If you’ve been doing this for any significant amount of time, you have a story about this. Something wasn’t instrumented correctly, some assumption about how metrics interact was incorrect. The test ended up going to the wrong users… something. This is still code and code is never perfect.