Online media photo created by creativeart — www.freepik.com

How to restrict and track Linux user outbound connections

Restricting outbound (egress) connections is an essential part of network security. Many articles explain why this is so important and how to do it overall. In simple words — you can have a super secure application and environment, but you can’t avoid using third-party software which may be compromised. In most cases, such software is not harmful until it downloads some helper tools, which can download backdoor, scrape and send sensitive information, etc. Without the possibility to upload/download stuff, malicious software in most cases can be disarmed.

It’s also the case where a vulnerability existing in a piece of software which can allow access into your environment by a hacker exploiting that vulnerability, if then they can’t easily contact the internet it makes it much harder to make lateral movements.

Simply blocking connections to outside is not a solution. There are lots of services that are dependent on external connectivity to do their job. For example — your OS needs downloading updates, so at least this requires DNS and HTTP/HTTPS connections.

For now, let’s imagine we have a Linux server with an unprivileged user.

Unprivileged — means user with /usr/sbin/nologin shell and other limitations. It’s used to run the Docker containers in a more secure environment.

For this user, for example, outbound connectivity will be blocked by the firewall, but not everything. Let say we want to make an exception for DNS, HTTP, and HTTPS traffic. While DNS requests can be easily restricted and tracked with a firewall, HTTP(S) is more complex and also encrypted.

So for this, we will use a proxy, built with Openresty with the HTTP(S) proxy CONNECT module ngx_http_proxy_connect

Openresty is an enhanced version of the Nginx web server and ngx_http_proxy_connect module provides support for the CONNECT method request. That’s how it is described in RFC7231:

The CONNECT method requests that the recipient establish a tunnel to the destination origin server identified by the request-target and, if successful, thereafter restrict its behavior to blind forwarding of packets, in both directions, until the tunnel is closed.

This method must be supported by both — the proxy and the client. And the client is a tricky part. Not all clients support this method.

For example, wget — a popular tool to download content from the web, doesn’t support it, especially the Busybox version used in Alpine Linux. Fortunately, this is rather an exception and most modern tools work just fine. In this case, wget can be replaced by curl.

But let’s get back to our Linux unprivileged user, we need somehow to enforce all his outbound connections to go only through the proxy. This can be achieved with the most common Linux firewall — iptables, and this is the first part of our setup.

We will create a new chain:

$ iptables --new-chain SANDBOX

route all traffic from that user to the sandbox chain

$ iptables -A OUTPUT -m owner --uid-owner john.doe -j SANDBOX

now in sandbox chain we care only about new connections, everything else can go

$ iptables -A SANDBOX -m state ! --state NEW -j RETURN

allow local subnet (put your subnet mask here)

$ iptables -A SANDBOX -p tcp -d 172.16.0.0/12 -j RETURN
$ iptables -A SANDBOX -p udp -d 172.16.0.0/12 -j RETURN

allow DNS from Cloudflare

$ iptables -A SANDBOX -p udp -d 1.1.1.1/32 --dport 53 -j RETURN

and log everything else, that came to this point

$ iptables -A SANDBOX -m state --state NEW -j LOG --log-prefix "Filtered outbound connection: "

then drop it

$ iptables -A SANDBOX -j REJECT

now we can try to run a sample request and see what happens

$ sudo -H -u john.doe curl -v ifconfig.me
* Trying 34.160.111.145:80…
* TCP_NODELAY set
* connect to 34.160.111.145 port 80 failed: Connection refused
* Failed to connect to ifconfig.me port 80: Connection refused
* Closing connection 0
curl: (7) Failed to connect to ifconfig.me port 80: Connection refused

so if there’s an outbound connection attempt to something not allowed in the firewall and not using the HTTP(S) proxy, in /var/log/syslog you will see a message like this:

kernel: Filtered outbound connection: IN= OUT=ens5 SRC=172.16.0.5 DST=34.160.111.145 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=16088 DF PROTO=TCP SPT=42218 DPT=80 WINDOW=62727 RES=0x00 SYN URGP=0

With this information, it’s easy to catch what else should be whitelisted or routed through the HTTP(S) proxy and also (what is even more important) — track if something is trying to connect outside which is not supposed to do so.

The second part is the proxy setup. We have our ready-to-use Ansible role https://github.com/Quiq/egress-proxy

Once Ansible setup finished on the server, let’s login there and see what we can do with it

We set environment variables http_proxy and https_proxy to 127.0.0.1:3128 which is our egress-proxy container, and curl seeing these variables will use them to access the proxy and connect to the destination host with CONNECT method

At the end docker logs -f egress-proxy command will show us two connections opened through the proxy.

In our Ansible role we have whitelist.yml with the list of domains allowed to connect and ifcomfig.me is there. Others would get403 — Forbidden

Now if we run our docker containers under this user, we can be sure, that our system has one more solid layer of security, all outbound connections are restricted and all connection attempts will be logged in the system log.