A Practical Guide to Amazon Route 53 Resolver DNS Firewall

Cloud architects and security engineers working on projects built on AWS put a lot of focus on threats coming into a network, and rightfully so. Bots, scrapers, and hackers are constantly trying to break into whatever they can. To counter those efforts, we deploy AWS WAF web ACLs, configure Security Groups and NACLs, and protect resources with AWS Shield Advanced. Those tools work well for inbound threats, but what about threats which leverage outbound connectivity? Sometimes not enough attention is paid to outbound filtering. AWS has a few tools to help in this regard. This blog post is focused on leveraging Route 53 Resolver DNS Firewall. First off, what security problem does this service solve?

Say a piece of malware gets on to an instance you have running in a VPC. It just barely managed to sneak by the ingress filtering you already have in place. It can’t do much on its own, so it needs to phone home to get more malware and some further instruction about what to do on the target system. To accomplish this, it calls out to a URL of a compromised domain. It is then able to download the payload and start its work. Route 53 Resolver DNS Firewall can help prevent malware from being requested from compromised or malicious domains.

To take the example a bit further, suppose the malware harvests some particular pieces of data that it encounters, perhaps passwords it intercepts. It needs some place to send them, so it starts to exfiltrate them using a domain controlled by a hacker group. It makes DNS requests to the domain using the data it has collected as subdomains. It requests user.Fred.pwd.Fr3dRocks.example.com, user.Sally.pwd.Abc123.example.com, etc. This type of attack is called DNS tunneling and has been around for a very long time. The hackers would look at their DNS server logs for example.com and be able to harvest sets of usernames and passwords exfiltrated as DNS requests. Route 53 Resolver DNS Firewall can stop this kind of activity.

Now that we’ve identified two practical use cases for this service, let’s set it up. You can approach this is one of two ways: explicitly blocking domains or explicitly allowing domains. The latter will provide the best level of security, but can be more challenging to properly configure, depending on the nature of your expected DNS requests from systems running in your VPC. If you choose to explicitly block domains, you can either define a list of domains you want to block, you can leverage AWS-managed lists, or a combination of both.

Assuming you’d like the allowlist approach for best security, the first step is to learn what DNS requests are coming from your VPC. This involves configuring Route 53 Resolver DNS Firewall, setting up logging to CloudWatch Logs, associating the firewall with with your VPC, and setting your allowed domains to *. This will send each and every DNS request to a CloudWatch Log Group. Assuming you have a steadily running workload in your VPC, let the logs collect for a couple of weeks. After that, you can use CloudWatch Logs Insights to query your Log Group. When run against your Log Group for an appropriate time period, this query will output a list of DNS requests with the number of times they were made, sorted from greatest to least:

fields @timestamp, srcaddr, query_name | stats count() as count by query_name | sort count desc

When I did this recently, I noticed some interesting things. We have an Amazon Aurora database in our VPC and a cluster of webservers that reach out to it. We saw hundreds of requests for the hostname of the Aurora database, but with random letters capitalized like this:

aUrOrA-PRImARy-Cluster.cluSTer-D2hbYfcAe7qn.uS-eAST-1.RdS.aMaZonaWs.cOm.
Aurora-prImarY-ClustEr.cLuSTer-D2HbyfCaE7QN.US-EaSt-1.rdS.AmazONAWS.COM.
aURoRa-PRimARY-CLUsTer.cLuSteR-d2Hbyfcae7qn.us-EAST-1.RDs.AmazONAws.Com.
AuroRa-prImARY-cLUStER.CLusTEr-D2hbyfCAE7Qn.Us-eaSt-1.RDs.aMazonawS.cOm.
AUroRa-PrImARy-CLUStEr.CLusTeR-d2HbYFCAe7qN.US-EaST-1.rDs.AMAzOnAWs.cOM.
aURoRA-PRIMArY-CLuSTER.ClusteR-D2hByFcAE7qn.US-eaSt-1.rdS.AMazONaWS.cOM.
auRORa-PrImary-ClUster.CluSter-D2hBYfCaE7Qn.us-eaSt-1.RDS.AMazoNAwS.CoM.
AuroRA-PriMARy-ClUStEr.clusTEr-d2hBYFcAe7qN.us-eAsT-1.RDs.amAzOnAwS.cOM.

The best I can figure, it’s a DNS cache-busting strategy related to the Aurora failover mechanism. In any case, we whitelisted *.amazonaws.com anyway, so there’s no need to pay close attention to this. I just thought it was odd :)

If you have systems that reach out to third-party sites to download patches or app dependencies, you’ll notice a lot of DNS requests for those domains, You’ll also probably notice requests for content delivery networks (CDNs). Many DNS names are actually CNAMEs for CDNs or other sites. It’s important to note that you not only have to allow-list the primary domain your system calls out to, but any other subdomains in its CNAME chain.

Once you build your initial allow-list from domains that appear in your Resolver logs and are confident you’ve got them all, you can flip your Route 53 Resolver DNS Firewall to BLOCK mode. When in BLOCK mode, you’re protected from the two types of attacks illustrated earlier.

One very important thing to do is to set up a metric filter for your Resolver logs. You’ll want to be alerted whenever there is a BLOCK. This will serve two purposes: to know if malware is attempting to phone home or exfiltrate data and to know if there are legitimate domains being blocked. This can happen if a site changes its CDN, or if there’s a new business requirement to reach out to a domain, etc.

Bearing in mind these points while following one of the many getting started guides out there will ensure an optimal experience with Route 53 Resolver DNS Firewall!