What is Bernie Sanders A/B Testing?
2020 update: How 2020 Candidates Use Behavioral Science to Convert Visitors into Donors
I thought this was going to be an in-depth analysis of what all the presidential candidates were testing on their websites (images, headlines, etc), but it turns out only Bernie Sanders is taking this whole “website conversion rate” thing seriously.
Other candidates may be testing (for example, Hillary has Optimizely on her site and does a good job of archiving experiments), but I did not find evidence of serious a/b testing.
Why would I spend part of my day on this?
Ever since I heard Optimizely’s origin story, I’ve been fascinated by politicians running a/b tests. It’s been done for ages with TV ads, direct mail, and phone calls. And now it’s become the norm for candidates to religiously a/b test their sites.
Turns out, not so much.
Instead, Bernie is the only candidate working hard to generate donations from his website and find out what messaging/imagery works the best. Makes sense. He’s a man of the people and he’s depending on his website to deliver.
So what’s Bernie testing?
It took some effort to go through Bernie’s testing code and extract the different variation code that’s still visible. I’m a curious guy. And a conversion geek of sorts. So here are a few highlights of what Bernie’s been testing, organized by theme:
Imagery: social proof FTW!
Here’s a clean image isolation. “Bernie at a rally” vs “Old school Bernie” vs “Close-up Bernie”. The question here seems to be which photo is more endearing or will best connect with visitors.
Below are a series of images used on the homepage. It appears as though social proof is winning against “lonely Bernie”. Note that I got the first image from the Way Back Machine, so the font is incorrect.
Also note that the copy is evolving along with the imagery. I’ll get more into copy down below…
Color: red, blue, or green?
You’ll note in the homepage example, above, that the CTA button color has evolved from red to green. Never bad to test.
Color was the main focus of another test on the Birthday campaign form, as seen below. Based on the Optimizely experiment ID, it’s possible to assume that this experiment was run AFTER the image isolation further above. That means that “Bernie at a rally” was the winner! Cool! Social proof wins again.
I’d be curious about using something other than a ghost button as the CTA…
Headline Copy: less desperation, more social proof!
Although most a/b testers focus on CTA size and color, words are where the money’s at. Also, the insights you can generate through simple headline tests can far outweigh the value of running design-focused tests.
Here are a list of headlines, in no particular order that have been tested by Bernie’s team on the homepage:
“We need your help…”
“Stand With Bernie”
“Make History with Bernie”
And on his splash page, where the focus tends to be on issues, the headlines cover various themes. The question is which issue really sticks? This is where personalization can be HUGE. I really hope they’re hooked up to a DMP that’s helping make these choices.
“If the environment were a bank, it would have been saved by now.”
“Our economy is rigged and our political system is corrupt.”
“The Koch brothers shouldn’t be allowed to buy our politicians.”
“Nobody who works 40 hours a week should be living in poverty”
There was one experiment to highlight that focused entirely on the issue of poverty, using longer vs shorter headlines:
“As the wealthiest country in the history of the world, it’s disturbing that we have one of the highest rates of childhood poverty.”
“The United States is rich enough to make sure none of our children live in poverty.”
“We’ve got to eradicate childhood poverty.”
Supporting Copy: negative social proof?
The 2–3 sentences after the headline have been tested as well. This is where the appeal to donate is really being made:
The “if everyone donated” subhead. It appears be working as it’s the one consistent element I’ve seen on the site since late last year. It does seem to contradict Roger Dooley’s characterization of Wikipedia’s pleas for help as being “negative social proof” (by pointing out that most people who visit the site DON’T donate to the campaign, thus creating a majority around non-donaters). Would be curious about testing “Join X,XXX,XXX other visitors to this website who made a contribution to stand up to big money…” wink-wink.
Long version:
If everyone visiting this site donated, Bernie would have more resources than all the billionaire-backed candidates put together — and he’d use them to get money out of politics forever. Take back your country from the billionaire class.
Short version:
They have the money, but we have the people. If everyone who visits this website joins our movement, there’s nothing we can’t accomplish together.
My recommendations (for Bernie’s eyes only)
It’s easy to give recommendations from afar, in the comfort of this Vancouver coffee shop. But I want Bernie to succeed and so will take this opportunity to offer a few points of advice, if I were to run his conversion optimization:
- Test against the current “negative social proof” subhead, if it hasn’t already been done. Rather than “if everyone who visits this website joins our movement,” talk in terms of “join the millions who’ve already made a contribution.”
- The donation amounts table. Unless there are legal reasons against testing it, I would test the s*** out of that thing. Check out this massive list of great ideas. Namely, I’d look at showing amounts that aren’t as nicely rounded and anchored higher. I suspect that testing here will be tough due to it being hosted on a 3rd party provider. DOH! 3rd party providers are the kryptonite of well-meaning conversion optimizers. But there is ALWAYS a way…
- And finally, website speed. This isn’t something that should be tested, but rather fixed. The site is super sluggish and has a crazy amount of font flickering. Paradoxically, slow custom font loading has been known to help with conversions in some instances, but wouldn’t count on it.
Have any other ideas for Bernie? Share them!
