Resources to Evolve Your A/B Testing Skills

This year at MozCon I am giving a talk about how growth minded marketers can evolve their A/B testing skills in three key areas: ideas, execution, and team. Over the course of the 45 minutes, I lay out how marketers can evolve from generating ideas to forming hypotheses; from executing and getting sh*t done to automating the process; and finally from having a team of people who work with you to advocates who support and promote you. I mention a lot of tips and resources in the talk but since I don’t have time to go into all of them, I am aggregating them all here in a post for easy reference.

Luckily, extremely intelligent people have been writing about what it takes to evolve and up level your skills. Here are some of my top — now it’s up to you to make your evolution a reality!

Ideas to Hypotheses

To evolve from generating a/b testing ideas like “let’s move this; let’s change that; how about we remove all of it” to well-formed, researched hypotheses, we need to collect 2 things: analytics and voice of the customer. Here are a list of resources I suggest you read and bookmark to become a hypothesis pro.

Read these articles to evolve from ideas to hypotheses.

For developing hypotheses:

• How to Write A Solid Hypothesis

For analytics:

• 10 Google Analytics Reports That Tell You Where Your Site is Leaking Money

For voice of the customer/human:

• Qualitative Questions that Lead to Great A/B Tests
• Conversion Equation by Oli Gardner
• Lift Model from WiderFunnel
• Psychological Triggers (but beware of backfiring)

Stories and Hackathons

• How to Host an A/B Testing Hackathon
• Case Study: MVMT Watches
• Case Study: Hillary Clinton Campaign Credit Card Details

Execution to Automation

Evolving from execution to automation means taking your “get-shit-done” hat off. Sometimes, you have to slow your roll to set yourself up for success. When you approach A/B testing from an automation perspective, you standardize the logistics and nitty gritty specifics first. The nitty gritty includes prioritization framework, stopping criteria, audiences and success metrics. With those parts nailed down, you can spend more time thinking about the substance of the test and the user experience.

Read these articles/download these materials to evolve from execution to automation.

For prioritization:

• A Method for Prioritization That Won’t Hurt Feelings

For defining goals:

• eBook: Define Your Company’s Data DNA

For defining stopping criteria:

• Evan Miller’s calculator
• Optimizely’s statistics engine: explained

For defining audiences:

• Optimizely Personalization Toolkit: Audiences Worksheet (shameless plug: I wrote it. Solid foundation for defining and identifying audiences here)

Stories

• Case Study: Brooks Running
• Automating Like a Human from Help Scout ❤

Team to Advocates

A team of humans who works with you to execute or automate your tests is fantastic. If you have this, you’re doing better than a lot. BUT why stop there. The next level of team is advocates: people who want to promote your work and help grow the focus on data-driven decision making.

Read these articles and download these materials to grow your team into advocates.

• 6 Clever Nudges to Build a Culture of Experimentation
• Results Sharing Template from Optimizely’s Testing Toolkit
• How to Get Everyone Involved in Optimization
• Making Optimization a C-Level Priority

The only constant is change

I hope you are always moving forward and only taking small steps back to make big leaps forward. I’d love to discuss all of these topics more so please reach out or at least comment with the articles/resources that have helped catapult your testing skills.