What Are Jeb Bush & Others A/B Testing?

(Not that anyone cares)

Alhan Keser
Feb 14, 2016 · 3 min read

This is quick follow-up to What is Bernie Sanders A/B Testing? in light of the GOP Debate tonight.

As I did previously, I took a look at the GOP candidates’ websites for any signs of A/B testing. Found some!

Here’s what Jeb’s been up to:

“Visible GW” vs “Not-so-visible GW”

“GW in full color” vs “GW less visible”

Well, this is awkward. As part of a signed baseball offer, Jeb’s team tested a full-color photo of GW vs one where he’s a lot less visible, as seen above. Okay, let’s focus on the form. It’s the form they were testing. Putting more focus on the form.

Jeb had another experiment worth mentioning, focused on the creation of a password during the donation process:

Password creation explanation text

This one is interesting, for a change. Jeb’s team is looking at whether clarifying that creating a password is optional helps or hurts.

Also, worth mentioning, Jeb’s been running geo-targeted tests looking at creating messaging specific to different states. Good thinking by this team.

Ben Carson

Found a couple more a/b tests on BenCarson.com

Book vs Hat Offer

“Get a book” vs “Get a hat”

The above test appears to be looking at what free offer works best to motivating donations on the landing page. Is it Carson’s book, “Gifted Hands” or a campaign hat?

“Donate $2” vs “Donate $25 + Hat”

“Donate $2” vs “Donate $25 + Hat”

In this experiment, a few things are being tested: The copy below the headline is asking for $2 in one variation, and $25 in the other, offering a free hat. This is close to being an interesting pricing test. There is another variable in the second variation, which is in the pricing table ($15, $25 instead of $10, $20, $35), making the result of this experiment difficult to interpret.

Thus far, I still haven’t seen a structured approach to A/B testing being applied to any of the candidate websites, on either side.

Soapbox

Not interested in your hot takes

Alhan Keser

Written by

At the intersection of experimentation and data science. Now at American Express, formerly at WiderFunnel. Views are entirely my own.

Soapbox

Soapbox

Not interested in your hot takes

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade