I can’t deny that A/B testing, when used correctly, in an organization that truly has the resources, can be a profitable endeavor. More often, it’s individuals or small startups that really can’t afford to be A/B testing, and are prone to being mislead by the results.
How I increased sign-ups 20,730% (by NOT A/B testing)
David Kadavy

I like where you’re going here and have been having this debate with my friends for quite some time.

I wonder if you could elaborate on why individuals or small startups cannot afford to be A/B testing. Lack of sufficient traffic, etc.?

Just to be the devil’s advocate here: isn’t the promise of A/B testing a methodical way to learn — many small changes over time, lifts can come from unexpected changes, and aim for a mix of modest and ambitious hypothesis to test?

Moreover, is this potentially dangerous for mature products, enterprises, and just about anyone with a large userbase to wholesale adopt non-A/B testing strategy?

I suppose you might be suggesting that we move away from A/B testing as a crutch for product discovery & validation and leverage it more for launch/rollout?

I’d love to hear your thoughts as I personally believe a lot of growth and product work is now overwhelming mechanical in the pursuit of being objective and methodological — for better and for worse.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.