Resources to Evolve Your A/B Testing Skills

This year at MozCon I am giving a talk about how growth minded marketers can evolve their A/B testing skills in three key areas: ideas, execution, and team. Over the course of the 45 minutes, I lay out how marketers can evolve from generating ideas to forming hypotheses; from executing and getting sh*t done to automating the process; and finally from having a team of people who work with you to advocates who support and promote you. I mention a lot of tips and resources in the talk but since I don’t have time to go into all of them, I am aggregating them all here in a post for easy reference.

Luckily, extremely intelligent people have been writing about what it takes to evolve and up level your skills. Here are some of my top — now it’s up to you to make your evolution a reality!

Ideas to Hypotheses

To evolve from generating a/b testing ideas like “let’s move this; let’s change that; how about we remove all of it” to well-formed, researched hypotheses, we need to collect 2 things: analytics and voice of the customer. Here are a list of resources I suggest you read and bookmark to become a hypothesis pro.

Read these articles to evolve from ideas to hypotheses.

For developing hypotheses:

For analytics:

For voice of the customer/human:

Stories and Hackathons

Execution to Automation

Evolving from execution to automation means taking your “get-shit-done” hat off. Sometimes, you have to slow your roll to set yourself up for success. When you approach A/B testing from an automation perspective, you standardize the logistics and nitty gritty specifics first. The nitty gritty includes prioritization framework, stopping criteria, audiences and success metrics. With those parts nailed down, you can spend more time thinking about the substance of the test and the user experience.

Read these articles/download these materials to evolve from execution to automation.

For prioritization:

For defining goals:

For defining stopping criteria:

For defining audiences:


Team to Advocates

A team of humans who works with you to execute or automate your tests is fantastic. If you have this, you’re doing better than a lot. BUT why stop there. The next level of team is advocates: people who want to promote your work and help grow the focus on data-driven decision making.

Read these articles and download these materials to grow your team into advocates.

The only constant is change

I hope you are always moving forward and only taking small steps back to make big leaps forward. I’d love to discuss all of these topics more so please reach out or at least comment with the articles/resources that have helped catapult your testing skills.