I wrote a $9 million line of code
In December 2016 I implemented an Optimizely experiment that did this:
The site in question has never required visitors to bar when foo-ing, and in fact the desktop version of the site already said this quite prominently. But several months back we found that visitors preferred to register emails on mobile, coming back later to complete the foo process, and since then the mobile and desktop experiences had diverged.
This simple copy change raised conversions by 35%, which translates to 1,600 additional foo applicants per month. At an average value of $475 each, that’s over $9 million in extra foo per year!
If you’re still reading, I assume you want to learn the following:
How to write a $9MM line of code
Here are three simple guidelines.
1. Write one line that is really 20,000 lines
Here’s the line of code I wrote:
$(‘.hero h3’).after('<h4>You don't have to bar.</h4>');
2. Be unoriginal
Our $9MM experiment was simply the mobile implementation of messaging we had tested on the desktop version of the site. Wins come from iteration, not inspiration. Steal ideas from previous experiments, other pages and audiences, competitor sites, best practices docs, and test them all.
3. Write thousands of worthless lines first
The $9MM experiment was the 96th we ran in 2016. We had other wins, but clearly we built losers and inconclusives as well. Nothing wrong with that — losers tell us what our visitors don’t respond to, and inconclusives remind us that we don’t know what matters until we test it. Always be testing.
If you run a hundred experiments a year, you’ve probably already had some embarrassingly simple wins. I’d love to hear about them. If you’re running 99 or fewer, let’s talk about how to turbocharge your experimentation program.