Supercharge your deliverability with Automated Testing Down the List
Action Network’s leading deliverability gets even better!
At Action Network, we spend a ton of time, publish a lot of blog posts, and host many trainings on email deliverability. That’s because if your supporters don’t see your email, they can’t sign your petition, donate, or RSVP for your event. Good email deliverability is critical to your mission.
Together with our core development partners Daily Kos, AFL-CIO, and the DNC, we’re always thinking of ways to help our partners improve their deliverability. Our newest feature, Automated Testing Down the List, helps organizations with large lists institute deliverability best practices on every email send, automatically.
What is testing down the list?
Testing Down the List is an advanced strategy for improving your email deliverability. As an email writer, your goal is to get your email to as many recipients on your list as possible while still maintaining a healthy open rate (above 15%). Typically, this means sending your emails to engaged activists — recipients who have opened, clicked, or taken action from an email recently, such as in the past 30, 60, or 90 days. Sending your emails to your most engaged activists is your safest bet to maintain good deliverability.
The downside to this strategy is that it limits your sending universe significantly. You may have thousands of activists on your list who subscribed at one point but haven’t opened an email in a while, and if you only send to your actives, you’ll never have the chance to re-engage them.
That’s where Testing Down the List comes into play. Sometimes you have an email that is good enough to re-engage your list while still maintaining a safe average open rate for the day. Testing Down the List allows you to test your email with different cohorts of actives without risking your deliverability. If the test goes well, you can send the email to a larger universe and effectively reengage folks on your list.
“Action Network is clearly built by people who get digital campaigning, and they’ve baked best practices right into the platform. We started testing down the list manually after joining a webinar hosted by the Action Network team, and we’ve seen results — now this new feature will streamline that process even further.”
— Jack Mumby, Deputy Digital Director, Common Cause
Previously, to do this testing you’d have to create different cohorts of ‘actives’ on your list, such as 30-day actives, 60-day actives, 180-day actives, and so on. Then, you’d use Action Network’s random limit to test each email to a portion of your first tier of most engaged people (your 30-day actives). If the email performed well with that audience, send it to the rest of the first tier and send it to a portion of your second tier of most engaged people (your 60-day actives). If that does well, send it to the rest of the second tier and a portion of the third tier (your 180-day actives), and so on.
Now, with Automated Testing Down the List, we’ve (you guessed it!) automated this process on Action Network.
How does Automated Testing Down the List work?
Our new Automated Testing Down the List feature makes this method much simpler. You can now test how your email will perform among activists who were active on your list with different built-in cohorts of actives (30 days, 60 days, 180 days, and so on) and send the “golden” email — the one that’s good enough to re-engage activists while still keeping your open rate high — to activists in those cohorts.
For example, with Automated Testing Down the List on Action Network, you could send an email to 1,000 random activists in each cohort (30-day actives, 60-day actives, etc.), see how each cohort performs (open rate, click rate, etc.), and then send the email to the rest of the cohorts you determine are performing well.
This kind of test is great for maintaining deliverability, among other uses — instead of unsubscribing inactive activists from your list or only targeting email to more active segments, you can test to a small random sample in each cohort and send the email just to cohorts you know will interact with it.
Note: For this test to work, you need to have a large number of activists in each cohort — we recommend that you use a random limit of at least 1,000 and have a list with over 10,000 activists in each cohort. Lists smaller than that will not give you usable test results.
How do I set up an automated test on Action Network?
Click the “Automate Testing” button on the Preview & Send page to set up an automated test to different cohorts on your email list.
After you click the Automate Testing button, Action Network will automatically split the target universe you selected up into six cohorts depending on activity and engagement level: 30-day actives, 60-day actives (activists who were active in the last 60 days but excluding activists in the 30-day actives cohort), 90 days, 180 days, 365 days, and the rest of your targeted universe. Once those cohorts have finished calculating, you’ll be able to see the counts for each cohort, set a random limit (for example, only send to 1,000 random activists in each cohort instead of the full list), and send your test.
After you hit send, you’ll be able to see how each cohort performed (opens, clicks, actions, donations, etc.) and choose which cohorts you’d like to include in a winner email. Choose the cohorts, click the ‘Choose Winner’ button, and you’ll have a copy of the email targeted to activists within your initial target universe, minus those who received the test, but just within these cohorts (and without a random limit), allowing you to send to the rest of activists within the cohorts you choose.
What if I want to automate that last step of choosing the winner and sending to the rest of my activists?
You’re in luck! You can choose to fully automate the test by choosing the “auto-win” option. With this option on, after sending the tests to all cohorts, we will wait a certain amount of time, chosen by you, and compare performance in each cohort with a statistic (such as opens or clicks) and a percentage that you also choose. A winning email will automatically be created and sent when that time expires, and any cohort that meets or beats that threshold will be automatically targeted in that winning email. Activists in the test email will be excluded, and the random limit will be removed.
Auto-Win can be set before you send the tests or after, while looking at test results. Auto-Win can also be toggled off later, assuming the winning email has not been sent yet.
Who has access to this feature?
Automated Testing Down the List is available to all partners, but we recommend that you only use this feature if your email list is large enough to get usable results — random limits should be at least 1,000, and you should have at least 10,000 activists in each cohort. That means your list should have at least 60,000 activists on it (10,000 per cohort, 6 cohorts).
Want to learn more? Join our live training on Thursday, June 25 at 2pm ET — RSVP here. In the meantime, you can dive into the details in our help documents.
Thank you to our Platform Development Committee members Daily Kos and AFL-CIO for building this exciting new feature with us! Through our cooperative development model, we’re focused on building tools with the progressive movement, for the progressive movement. Read more about our Progressive Approach to Movement Tech here.
