A/B Testing An Email — The Body Issue

George Abraham
Oct 7, 2019 · 4 min read
Image for post
Image for post

This is the second post in a 3 part series about A/B testing an email. The first post was about making a good first impression.

After an Open

If you want to iteratively test on the design of an email you need to design scalable templates and instrument your email code in a manner that is easily testable. One approach is to identify areas in the email body that can be leveraged for experimentation. It is important to remember that specific levers may work well on some populations while they may not on others.

Scalable Design

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
A win back campaign from the New York Times

Unpacking the design

Image for post
Image for post
Image for post
Image for post

What's important to note here is that the design is built to test any of the defined levers . Testing these levers at scale in a high-frequency campaign could be rewarding. At the very least you will learn about what matters to your audience.

In the example above the design is able to support two very different tactics.

Headline 1 for the message on the left is a FOMO play where the fleeting nature of the offer is highlighted — just 24 hours. The message on the right, however, is leaning on a value proposition of the newspaper — clarity.

In both cases, the hero image is easily extended to make sense for the main tagline. The template design is scaleable as it can support visual and tactic variations.

Finally, the text-heavy module on the bottom is used to support the theme of the message.

In the message on the left, the text is reiterating the Subscribe Soon tagline, whereas the message on the right teases that the paper will provide clarity around purchasing a car without regrets.

A note about Footers

Whether or not you plan to experiment in this area, you should invest in creating scalable footers across the various dimensions of your messaging program (account, marketing, etc).

If there is one thing that is absolutely important in the footer, it is to give your customers a one-click unsubscribe experience. Nobody wants to enter their email again on your preferences page or “click ok to confirm”. If customers are not engaging with your email, you really want to know right away so that you can adjust your messaging program.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Conclusion

In the next post in this series, we will discuss the landing experience from an email.

The Notifications Blog

All about Notifications

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store