DNS Market Share Analysis — Identifying the Most Popular DNS providers
There has never been so many DNS options for us to use right now. From Google's 8.8.8.8, to Quad9, OpenDNS, CloudFlare, CleanBrowsing or even the default DNS provided by your ISP , there is a lot going on. Including new protocols like DNS over HTTPS (DoH), it seems that we are finally getting to a point where DNS security is going to be taken seriously.
Within the tech community we assume that most people are using one of those big players and their privacy options enabled. In fact, if I believed on what I read on Twitter, I would think that everyone switched to CloudFlare's 1.1.1.1 after they launched.
However, is this true? What are people really using for their DNS resolution?
Measuring DNS utilization
The only way to know is by looking at real data. Every time you try to do a DNS resolution, your DNS resolver (say Google's 8.8.8.8) will contact the authoritative DNS for the domain you are trying to visit.
For example, if I am trying to visit example.com using Google's 8.8.8.8, that's a summarized flow of what would happen:
mycomputer — > Google's 8.8.8.8 — > Authoritative DNS for example.com
My computer would contact Google's that would contact the NameServer for example.com in my behalf. If I look at the DNS query logs for example.com , I can identify all the DNS resolvers being used and find the most popular among them.
But that only gives me a partial view for that specific domain. If I wanted to get visibility for the whole Internet, I needed a lot more domains and a lot more data… What if I could get access to the logs for thousands of domains to get a good sense of the most popular DNS providers across the Internet? Maybe all I had to do was to ask. And ask I did and access I got.
I spoke to a few colleagues online about that and one of them offered to give me access to anonymized logs for the hosting provider that he works for (with permission of his boss). He host the authoritative DNS for over 11,000 domains — from local shops to blogs and even some porn domains. A nice mix for our analysis.
He removed all data from the logs, except for the resolver IP address that was contacting his DNS servers. The log dump had 30,485,500 DNS request for the course of a many hours of data. That should be enough for our analysis.
Most Popular DNS resolvers
We analyzed the logs and categorized them by the unicast IP of some of the most popular DNS resolvers. We built a list based on what we thought would be the most popular ones:
DNS Providers
* Google (free + unfiltered)
* OpenDNS (free + blocks malicious domains)
* CloudFlare (free + unfiltered)
* Quad9 (free + blocks malicious domains)
* Norton (free +blocks malicious domains)
* CleanBrowsing (free + blocks adult / porn content)
* NeuStar (free + unfiltered)
ISP-based DNS Providers
* Comcast
* Frontier
* Cogeco
* Vodafone
* Aol
* Telefonica
* Telus
* T-mobile
* ATT
* Bell
* Cox
And tried to categorize the request based on it. However, it did not work as well. There were a lot of different IP addresses per provider and multiple networks involved, so we switched tactics and used the reverse IP + IP whois to identify the provider. That has some limitations ,but we feel gets us very close to what we needed.
Results
The results quite didn't match our expectations at all. First, only 15% of all DNS requests were using one of the most popular DNS providers:
#1 Google: 13.21%
#2 OpenDNS: 1.82%
#3 NeuStar: 0.15%
#4 Quad9: 0.13%
#5 CloudFlare: 0.12%
Google was the number 1, with 13% of all DNS traffic coming through them. OpenDNS was the number 2, with close to 2% of all DNS traffic. Quad9 and CloudFlare were close to each other with around 0.12% of the traffic. Not bad for CloudFlare that has just launched -but still very far from OpenDNS and Google.
I had to remove Norton from the numbers as they use the NeuStar network, so all requests from Norton users were showing up as NeuStar. CleanBrowsing numbers were also hard to track as they use multiple networks, so I removed them from the list.
Datacenter-based Traffic
#1 Amazon: 7.71%
#2 Facebook: 4.21%
#3 Microsoft: 3.28%
#4 Apple: 2.10%
#5 Linode: 0.85%
#6 Digital Ocean: 0.79%
#7 Twitter: 0.63%
The other piece that surprised me was datacenter-based traffic. Almost 40% of all DNS requests came from datacenters resolvers. Things like Amazon AWS, Azure, Linode, Apple, Facebook, etc. The number #1 was Amazon with almost 8% of all DNS traffic coming from different AWS instances.
I assume that those requests were generated by bots and crawlers. But interesting to see how such a high percentage of the Internet traffic is all "fake".
ISP-based DNS Providers
#1 Comcast: 2.36%
#2 Cogeco: 2.04%
#3 T-Mobile: 1.97%
#4 Claro: 1.83%
#5 Telefonica: 1.49%
#6 Aol: 1.48%
#7 Charter: 1.46%
#8 Vodafone: 1.40%
#9 Qwest: 1.30%
#10 ATT-Mobility: 1.21%
#11 Frontier: 1.20%
#12 Sprint: 1.17%
#12 Cox: .98%
#14 Verizon: 0.90%
#15 Orange: 0.67%
#16 Rogers: 0.49%
#17 Shaw: 0.45%
The rest of the DNS traffic came from residential (or business) ISP resolvers. They were very well spread out across all the major ones in the world. The list goes through hundreds of small ISPs all generating a small percentage of the traffic. Not as decentralized as I would have imagined.
Whatever DNS Comes By Default
As I said, these numbers surprised me. I assumed most people would be using a custom DNS provider for their Internet, but that's far from the truth. In fact, I think the majority just uses whatever comes by default from their provider. And the only reason people are using Google's 8.8.8.8 so much is because many devices come with it pre-configured. I wonder if OpenDNS is also being pre-configured on some Cisco routers… What you think?
**If you host authoritative DNS servers and is willing to share your anonymized logs with me, let me know. I would love even more data for my next research.
