Open in app

Sign in

Write

Sign in

Recursive DNS Resolver with AD-Blocking Features — Part 1

Wouldn’t it be amazing to have an home DNS server which filters Advertisements, Malicious Sites or other bad sites’ categories and recursively resolves the names without using official DNS servers like the one provided by Google, CloudFlare or OpenDNS, for example?

Gianni Costanzi
Nerd For Tech
15 min readApr 6, 2021

--

Updates:

  • 17 April 2021: added DNSSEC validation section

Provider DNSs

The majority of people uses internet at home without messing up with the configuration of the modem/router provided by the Telco and they use the provider’s DNS servers which are returned automatically by provider when the modem brings up the PPPoE connection when it is powered up.

Free public (and faster) DNS servers

The first step to have a better surfing experience at home is to connect to the router and change the DNS servers returned on the local LAN by the DHCP server replacing them with some public DNS servers like CloudFlare 1.1.1.1 which is very fast or one of its variants like 1.1.1.2 (no malware domains) or 1.1.1.3 (no malware and no adult content).

The advantage of using these DNS servers is that they are really fast compared to the ones of the providers. The disadvantage of using DNS servers like Google ones, is that you give google a lot of information about the domains you browse, because each time you visit www.mysecretsite.org you’re asking google “what’s the IP address of www.mysecretsite.org?”, thus providing it the whole list of domains you visit.

Recursive DNS Resolver@Home

The next step is to configure a Recursive DNS resolver in your home network.

A recursive DNS resolver does not asks public well-known DNS server to resolve Fully Qualified Domain Names (FQDNs), but instead it queries the root servers (which it must known in advance) asking them to which DNS server ask the top level domain of the FQDN. Then it goes on asking one of the DNS servers it has got with the next part of the domain and so on, so it acts recursively.

Let’s assume for example that we want to get the IP address of www.medium.com and do the recursive resolution “by hand”. I’ve got a list of root DNS servers from wget https://www.internic.net/domain/named.root

It contains for example the following server:

Here is how we can resolve the FQDN www.medium.com recursively:

The problem of recursive DNS resolution

If you simply enable a recursive DNS server that recursively resolves every FQDN, you would have a high performance hit in DNS resolution, because recursion requires a lot of time (you could pass from 20–30ms to 2–300ms for each query, or even more).

Caching and prefetching + serving expired 0-TTL resolutions is the way

If you enable an home recursive DNS, you definitely want to enable caching and prefetching:

  • Caching: the resolver recursively resolves an FQDN and then it stores it into it’s local cache for the amount of time specified by the TTL (5 minutes for the www.medium.com resolution above)
  • Prefetching: the resolver can proactively prefetch FQDNs in cache when you ask for them and the remaining life is under a certain percentage of the returned TTL
  • Serving expired records while resolving them: if devices in your LAN ask for www.medium.com every 10 minutes, there won’t be a resolution in the resolver cache since it has a TTL of 5 minutes. What the resolver can do is return an expired resolution with a TTL of zero (so that the Operating System of the device does not cache it) while starting an immediate recursive resolution for that domain. Recursive resolvers like Unbound support serving expired records and you can specify a maximum amount of time during which it returns an expired record if it is not able to resolve it recursively (usually you set it to 1-hour to avoid returning resolutions for removed domains for too long)

Avoiding the Advertisements.. and thousands of other bad sites

As I’ve already said above, there are some public DNS resolvers like Cloudflare for Families that can remove part of undesired DNS resolutions, for example the ones related to adult content if you want to have a safer browser experience for your kids.

What should you do if you want a much greater flexibility and you want to implement also a recursive resolution on your own at home? In this case Pi-hole is the solution: it acts like a blackhole for bad domains, thus the name -hole. I think the Pi in the name it is due to the fact that a Raspberry-Pi is a perfect candidate to run it into your home-network, but I’m just guessing :)

Implementing the Solution on a Raspberry-Pi (or equivalent)

Pi-hole Configuration

You can buy a Raspberry-Pi for very few bucks, you don’t need a new RPi4, an RPi3 like the one I have (and maybe also one of the previous versions) is perfectly fine.

You can install it following the guide here:

pi-hole/pi-hole

Network-wide ad blocking via your own Linux hardware The Pi-hole® is a DNS sinkhole that protects your devices from…

github.com

I won’t focus on every aspect of the configuration, I’ll just focus on the Ad-lists, the DNS setup and caching.

Once installed, you can set an admin password via the following command:

Then you can connect to the admin interface on http://RPI_ADDRESS/admin and choose Login on the left menu to login with the just set password.

First of all, you can change the DNS to which PiHole will forward queries by going into Settings -> DNS

Pi-hole DNS Forwarders Setup

As you can see, you can easily choose a primary and secondary IPv4 server among Google, OpenDNS etc, or you can set your own Upstream DNS servers on the right. Keep this in mind, since we will use the custom setup in order to forward queries to the Unbound Recursive Resolver.

Then there is the cache: if you just want to add the Pi-hole layer of filtering to the standard resolution to Google and OpenDNS, you should leave cache as it is. If you are going to use Unbound, with it’s own cache, DNSSEC and prefetching mechanism (which is triggered by client DNS queries) you must disable DNS Sec and Cache on PiHole.

In the same page where you’ve just setup Upstream DNS servers, you must disable DNSSEC if it is enabled (since it can be done by Unbound). I’ve also enabled the forwarding of reverse lookup of private ip addresses because it will be done by Unbound, otherwise if you use public DNS as upstream servers leave the option that blocks the reverse lookup for private ip addresses enabled, since it can leak information about your network to the outside.

Private IP Reverse Lookup and DNSSEC settings

Caching can not be disabled via GUI, it is necessary to disable it via CLI. So, edit /etc/pihole/setupVars.conf and set the following variable:

then restart pihole with:

Reconnect to the GUI and go to Settings -> System and have a look on the right side, you should see the DNS cache entries with all zeros (some caching still happens, but almost nothing and it is related to local resolutions)

By default, pi-hole blocks about 60k domains with the configured AD-List (at the time of writing, it may change in the future), but you can increase the amount of blocked domains by adding other lists. You can find more information on this useful blog post:

Unlock the full potential of Pihole

Foreword: I’m fascinated by technology and I wanted to share my findings while expirementing with Pihole. I’m not…

obutterbach.medium.com

Otherwise you can pick your own lists from FilterLists by filtering for lists which are compatible with Pi-hole.

Note: you can no longer add the lists you find in the blog post above via editing the adlists.list file via cli, now you must go via GUI in the Group Management menu on the left. I wanted to add the lists used in the blog post above and so I’ve created some grups with different categories in Group Manageent -> Groups:

Groups Management

Groups are useful because with a single click you can enable or disable an entire group of adlists, so I suggest to use them.

Then go to Groups -> Adlists and add some of the lists that you’ve found:

Adlists Management

As it is written in gray within the Address input box, you can paste more than one URL at once, they say space-separated urls but also urls with a carriage return like the one you find in the blog I’ve linked work. After you’ve imported the lists, you can change the group assignement by removing the default group and assigning the category you want. Then go to Tools -> Update Gravity (yeah, it is a blackhole, you are going to increase its gravity force :) ) and after a while (it can take several seconds) the list of blocked domains you can find in the upper right corner of your dashboard will increase from the original 60k domains to some hundreds of thousands like in my case:

Unbound configuration

Finally, we must configure the recursive resolver, to complete our DNS AD-Free & Recursive stack.

I’ve installed it on the same RPi3 of Pi-hole, so I will need to change the port on which it listens for DNS queries, since the standard 53/UDP port is used by Pi-hole (and we want it to be on the standard port, since our devices can’t be configured to use non standard ports for DNS queries). So, we will use 50053/UDP.

You can easily install unbound via apt:

Then get the root.hints file:

Then you can customize the configuration by editing /etc/unbound/unbound.conf

I’ll post here the result of uncommented lines in my configuration and add some comments inline to explain it better:

/etc/unbound/unbound.conf

Restart unbound with the new configuration:

you can test that unbound is working via the following command on the device where it is running (you can replace 127.0.0.1 with the IP address of the device if testing from another device):

As you can see, I’ve had a very bad performance for the first recursive query. Let’s repeat it again:

Wow, now it is in cache, and we have an immediate reply.

Pointing Pi-hole to Unbound

Now that we have a working Recursive DNS Resolver, go back into Pi-hole GUI, Settings -> DNS and configure Unbound as a resolver. In this case, it is running on the same system (127.0.0.1 means the local system) but on a non standard port so let’s go with the following:

Now, repeat the DNS resolution test with the commands above (remove the -p 50053 option because Pi-hole is running on the standard 53/UDP port) and test that everything works fine.

If everything is fine, go back to the Provider’s router DHCP configuration and replace the Google/Cloudflare/OpenDNS or other DNSs with the IP address of your Raspberry. Then you can reboot the modem to force your devices to reconnect to the local LAN and get the updated DNS from DHCP.

Verifying DNSSEC is working

You can perform some DNSSEC checks by connecting on the device running Pi-hole and Unbound (not querying them from the outside) and executing the following commands:

You can perform the check also with dig. As you can read in more detail on The DNS Institute page, if DNSSEC is working you should see (the following 3 points are taken directly from The DNS Institute page):

  1. The presence of the Authenticated Data (ad) flag in the header
  2. The DNSSEC OK (do) flag indicating the recursive server is DNSSEC-aware
  3. An additional resource record of type RRSIG, with the same name as the A record.

Note: I’ve used internetsociety.org as test domain because with www.isc.org as suggested on The DNS Institute page I couldn’t get the ad flag, so something is wrong with the validation of that answer but I’m not an expert so I’ve searched for valid domains to test DNSSEC and I’ve found this one on the DNSSEC Test Sites page on The Internet Society site.

The Internet Society page mentioned above shows also some sites with bad DNSSEC signatures that you can check. In this case with both dig and delv you should get a SERVFAIL in the output:

Monitoring Pi-hole

Pi-hole has it’s beautiful GUI that lets you monitor its performance, so go on the Dashboard and on the top bar you will see the amount of ADs your Pi-hole have blocked today. We always talk about ADs but as you have seen it can block also malware and other bad stuff listed in the adlists you’ve configured.

You can also query the following link to have a JSON output of internal stats. I’ve used it to send data to my InfluxDB2 server and to monitor Pi-hole performance via Grafana:

Pi-hole internal stats (JSON format)

Monitoring Unbound

You can get some info about how Unbound is performing by using unbound-control. Just launch unbound-control to see all the info you can ask for. We’re interested now in statistics so let’s run it with unbound-control stats_noreset (by default if you invoke it with stats the statistics are reset, unless you specify statistics-cumulative: yes as I’ve done in my unbound.conf ).

You’ll get something like this (I’ll omit the threadX rows that gives statistics per thread, in my case from thread0 to thread3 since I’ve specified to run with 4 threads):

I send also this info, after parsing it with Python, to my InfluxDB2 server.

Visualizing the performance of the stack

I won’t go into details of my monitoring panes in Grafana, it could be an interesting topic for a future post, but I can show you that my Pi-Holes and Unbound servers (yes, I have two Pi-holes and each of them points to the two Unbound servers, with a stack running on an RPi3 and the other running in Docker Containers on my Synology NAS, stuff for another post too :) ).

As you can see, we’ve almost no caching on Pi-holes:

Almost no caching on Pi-holes

Unbound gives us a lot of info:

  • Cache hits are over 96% of the number of queries received by both Unbound servers
  • We have a lot of prefetches, which is good because they increases the Cache hits
  • We also have a good amount of expired entries, which is the number of queries served via expired records (that are counted as a Cache hit)
  • What can’t be answered with the cache (even with expired entries) requires a recursive query/reply which is counted by the last entry below
  • We have no IP-ratelimited queries, and this is normal for a home network with few devices
A lot of Cache Hits and prefetch on Unbound, really good!

Note: you may ask yourself why the number of forwarded queries on Pi-holes is lower than the number of queries received by Unbound servers, but this is simply due to the fact that Pi-hole numbers are reset every day and soometimes I restart services (updates, device reboot etc)

Since Grafana is like a drug, when you have the data, you start plotting everything, here you can see some other stuff I plot about my redundant DNS stack :)

Note: utilization of Pi-holes and Unbound servers is not balanced since 1) the operating system tends to use the first DNS, which in my case is the Pi-hole running on Raspberry and 2) I think (but I’m not sure) that also Pi-hole tends to forward queries to the first DNS, or maybe it has some mechanism that sees the one that answers faster and tends to use it the most

DNS Resolvers Dashboard #1
DNS Resolvers Dashboard #2

Conclusions

This is my first new article here on Medium, I hope someone will find it inspiring and maybe useful, you can reach me on the social pages linked in my profile if you have questions or you can comment here :)

Update: added a follow-up story about installing Pi-hole on a Synology NAS with Docker with tips about how to disable caching in a dockerized environment (which is not straightforward as on a standard linux installation). You can find it on Recursive DNS +AD-Blocker — Part 2: Installing Pi-hole without caching on Synology NAS with Docker

Pihole
Unbound
DNS
Raspberry
Networking

--

--

Gianni Costanzi
Nerd For Tech

Written by Gianni Costanzi

67 Followers

Network Engineer, Music Lover, Motorbike Rider, Amateur Photographer, Nerd-inside

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams