How to configure systemd-resolved for querying internal domains on different DNS servers

How to use a private server for a private domain

Task: Configure systemd-resolved to send normal requests to default DNS server(s), but send requests for foo.internal only to server 192.168.1.1. Also, do not send any other requests (except for foo.internal) to 192.168.1.1

Expanded explanation

We have a server with configured DNS resolvers. We won’t focus on that part of configuration, it can be static, or coming from DHCP.

Now, we want to add our zone foo.internal , which is served by our servers 192.168.0.1 and 192.168.0.2, I added second IP for realistic reasons ). This server is responsible only for zone foo.internal and we don’t want it to be recursive resolver (e.g. we don’t want to see any requests for A for google.com coming to it.

We need to configure systemd-resolved on host to do so.

An obvious but incorrect solution

This solution will work, but it will cause multiple refusals from the private DNS servers and will slow down resolving..

/etc/systemd/resolved.conf.d/10_foo.internal.conf

[Resolve]
DNS=192.168.0.1 192.168.0.2
Domains=~foo.internal

What it will do:

• Add 192.168.0.1 and 192.168.0.2 to the DNS servers list for resolving.
• Send requests for foo.internal to those servers.

What it will ALSO do:

• Send requests for google.com to 192.168.0.1 , and if it fail, send request to the next server (192.168.0.2 ), and only after that, send to a normal recursive resolver server you have. Two refuses for each request, slowing down resolving (x3 slowdown!) and causing your private servers suffer with unjust useless traffic which they need to refuse.

Proper solution

The essential part of the proper solution is based on setting default-route option to ‘false’ for resolved. It is described here.

You CAN NOT set this option from resolved.conf.

Also, this option will kill a normal resolution (e.g. requests for google.com ).

So, we need to augment this solution.

Sketch (not a production grade solution):

ip link add resolver0 type dummy
ip link set up dev resolver0
ip address add 127.0.0.52/32 dev resolver0
resolvectl domain resolver0 '~foo.internal'
resolvectl default-route false
resolvectl dns resolver0 192.168.0.1 192.168.0.2

What it does?

1. Create a dummy interface with some throw-away IP address, bringing it up.
2. Set a special configuration for that interface: disable default resolving, enable ‘~’ resolving for domain foo.internal , set out custom DNS servers for it.

Why does it work?

1. We create an additional interface, for which we add some ‘special’ settings, while not touching out default interface with default resolving settings. Our normal resolution (e.g. google.com ) works as before, no changes at all.
2. Those ‘special’ settings intercept foo.internal and send requests via resolve0 interface and it’s special servers (192.168.0.1, 192.168.0.2)
3. We disable all other resolution on interface resolve0, therefore, it does not receive any requests for, e.g. google.com .

Why does it WORK?

From a DNS standpoint of view, everything is fine: normal resolving goes via the default route and to default servers, a special domain goes into a special interface. That’s what systemd proposes for VPN.

There is one last piece of the puzzle. Why requests to 192.168.0.1 send via resolve0 interface (type dummy) are working? How do they get from our phony non-routable 127.0.0.52 to 192.168.0.1, and, more importantly, how do the answers get back?

Maaaaagic….

When system chooses interface to send requests, it uses routing table to find which interface can reach the target. It can be slightly complicated if you have multi-home configuration or system without default route, but if you have default route, and it’s your single connection to the outside world, it’s simple: this interface is it.

Even you set rules for interface resolve0 , to reach 192.168.0.1, system (gai.conf, etc) can’t use 127.0.0.52 . So it selects your normal default interface, and uses it’s address as src IP for DNS request.

Therefore, systemd-resolved uses resolved0 settings to send requests for the private zone to the private server (and never send anything else there), but Linux routes this traffic in the best possible way, reaching those servers via available routes.

Production grade code

I’m lazy and won’t write playbooks for you for free. I gave you the solution, now orchestrate it into production code yourself.

Few things to consider.

First: you can’t put Exec into .netdev or .network units, but you can make .service unit with type=oneshort to BindsTo=resolved0.device

Second: you can try to combine all three resolvectl stanzas into one, but I found it can’t survive repeated runs and return an error, so keep them as three separate stanzas.

Third: Put private DNS addresses in different order on different servers (Jinja: dsn_list | shuffle(seed=inventory_hostname)| join(' ') ) to balance load on the servers.