Email reputation — how meaningful is sender score?

I’m writing this article because recently I terminated a contract with a large, well known and trusted business because they benchmarked their performance on a metric I don’t think they understood — while disregarding more valuable, pertinent metrics aka— harder facts. I wanted to remind businesses how flawed and foolish that can be.

An introduction… Domain Authority

The digital marketing world lives on data — numbers, KPIs, metrics. ROI, ROAS, CPA, CPM. We rely on numbers to benchmark progress, get feedback on activity and compare ourselves the competition.

Some of these numbers have been around for ever, but some come and go. Take the world of SEO — PageRank used to be one of the numbers people obsessed over. PageRank was a score from 1–10 that Google created and used in their ranking algorithm. If you worked in SEO you might have been focussed on increasing your PageRank, and getting links to your site from pages with a higher score than you. People buying and selling domain names might have used a higher PageRank as a reason to up the value of a domain.

But then on April 15, 2016 Google removed PageRank data from public view.

Today the key metric of importance I am seeing in the world of SEO is ‘Domain Authority’ — sometimes referred to as ‘DA’. Domain Authority is a score from 0 to 100 derived by Moz that predicts how well a website will rank. DA has become a well established and popular indication of site ranking and value — but we need to remember where it comes from: DA is a metric established by a third party business who sell SEO services.

If you never saw your brand on Google, if you had no traffic from Google — and if in webmaster tools you had no rankings — and a manual penalty — yet your SEO agency stuck to a line of ‘yeah, but you do have a DA of 80, so everything is OK’ — you’d be right in kicking off — and kicking that agency to the curb.

Moz’s DA has become a well established and popular indication of site ranking, elevating the profile and trust in their business. Great marketing tool for Moz, and great tool for SEOs to measure and benchmark.

Google Trends data showing the growth of ‘Domain Authority’ searches over-taking the dwindling number of searches for ‘pagerank’

The trouble is, useful as domain authority is, a problem happens when people forget that it’s a useful indicator, and start viewing numbers like this as hard facts. There’s something elegant about a single number that can cumulatively evaluate the value of something, but by simplifying we can start to miss the point and the big picture, and fool ourselves about true performance.

Moz update DA, to try and keep it up to date and as accurate as possible. But a DA of 80 (a very good one) does not mean you’ll rank position 1 for all your keywords. If you never saw your brand on Google, if you had no traffic from Google — and if in webmaster tools you had no rankings — and a manual penalty — yet your SEO agency stuck to a line of ‘yeah, but you do have a DA of 80, so everything is OK’ — you’d be right in kicking off — and kicking that agency to the curb.

That’s essentially the situation I got in with my email provider — and the metric they blindly followed inspite of hard facts — ‘Sender Score’

The email version of DA

Sender score is a third party metric that tries to demonstrate the reputation of an email sender or IP. A bit like Moz DA, it’s a third party metric to try and simplify something really complex — which also serves as a great marketing tool for their business (helping improve reputation and deliverability)

Now this isn’t a dig at Sender Score — I use it, yeh. But I also use other metrics — like my open rate, and Google’s postmaster tools.

I got in a situation where my open rate started to diminish. I emails my ESP — they said ‘look at your creative’. Thanks.

The situation worsens, I pay a premium to my ESP for more details reporting, and priority support. Their system reckons all is OK, and their priority support tell me to look at my creative — although much quicker this time.

The situation keeps on getting worse — investors in the business start getting our emails in their spam. I talk to other companies — ‘ditch that ESP, you’re on shared IP’s with a bunch of spammers probably’. I ask the ESP for a dedicated IP, do a ton of research — keep tweaking content, segments etc. — still no progress. (I came up with a ton of great ways to analyse email performance, I’ll have to share in another post some time!)

Then I discover Google Postmaster tools. 5 or so clicks later I have it — Google see’s our historical IPs, and our new dedicated IP as low quality — meaning that they will most probably filter us as spam.

This is what Google thought of our bad IPs — the red columns were when we were sending on an IP with a 98 sender score, the constant yellow is our new IP with a 99 sender score. The multi-colours bars was the old shared IP set up.

the metric they blindly followed inspite of hard facts — ‘Sender Score’

I share this exciting finding with my ESP. Their response — ‘oh but your Sender score is 98 — that’s really great, so there is not a problem our end’.

That was the moment they lost my business.

Despite many many lengthly conversations, sharing of screenshots — they ignored what I saw as the critical fact of ‘Google is telling me the IPs are bad’ — and they fell back on the 3rd party metric that suggests the IP is fantastic, ‘maybe you need to work on your content and strategy’ (Infuriating).

NB — Yeh, ofcourse I was working on content, HTML, strategy, engagement etc. No turn was left un stoned.

We hit a wall. So I met a great ESP at a trade fair. I told them all of this, and they chuckled. Contracts were signed, and now things are picking up. Our new dedicated IP has a 99 sender score, but more importantly, Google likes it much better too. And phew — the companies investors are seeing our emails again!

So what’s the take home?

Don’t only look at sender score — sign up for google postmaster tools — it’s free and awesome.

Also — remember where a stat comes from — and if email matters to you, don’t use the big shiny ESP that everybody else (including spammers) uses, they don’t care about you very much.

Also if you work for sender score, maybe re-calibrate things.. Here’s an analysis I did comparing sender score vs the IP reputation I got from Google. Google rates ‘low’ quality servers with a Sender Score of 98! (other with a score of 98 get rated ‘medium’ and ‘high’ suggesting poor correlation!)

Sender score of 99 looks safe, but 98 could put you in Google’s ‘low’ category!