Mike Green
Mar 20, 2010 · 9 min read

Last year I wrote a guide on how to use Shorewall as a firewall and router for Proxmox. As a follow up I will answer a few questions I’ve received about that guide that can help you extend its use.

Proxy ARP

The most common question is in regards to proxy ARP. Enabling this option will allow you to assign a public IP directly to your guest VM, eliminating the need for port forwarding (DNAT) or having to worry about the MAC address.

As an example use for proxy ARP, it is helpful for those using a a SIP-based VoIP server since a STUN server is no longer required.

Enabling Proxy ARP

The first step is to ensure that Proxy ARP is enabled. This is a fairly simple task and involves adding an single line to one of your static network stanzas. Which one precisely depends on your system setup; for those who have an eth0 stanza, you can use it there. For others who only have a vmbr0..n stanza, the additional line should be placed there.

iface eth0 inet static
# ... existing lines ...
post-up echo 1 > /proc/sys/net/ipv4/conf/all/proxy_arp

These changes will take effect on your next boot or whenever you restart your networking services.

Add a route to Shorewall

Shorewall needs to know that you’d like to use proxy ARP, for which IP that is and where this IP needs to be routed to. The beauty of Shorewall is its simplicity and so this can be quickly done by creating a file named /etc/shorewall/proxyarp that will contain this:

#ADDRESS    INTERFACE    EXTERNAL    HAVEROUTE    PERSISTENT
92.22.33.44 vmbr0 eth0

Translated, this informs Shorewall that the public IP 92.22.33.44 attached to eth0 needs to be forwarded to a guest VM (using this public IP) attached to the vmbr0 bridge.

As you can tell from the first line, there are two additional options: haveroute and persistent.

The haveroute option determines whether Shorewall should create a route from the external interface to the bridge. For guest containers based on OpenVZ, Proxmox will take of creating the route. But for fully virtualised containers (based on KVM), you need to create this route yourself. In this case we tell Shorewall to create the route for us, by keep this option’s value at its default value (blank, or No).

When the haveroute option is set to No (default), the persistent option tells Shorewall if it should keep the created route active if Shorewall is stopped. Generally, and for security reasons, you should leave this at its default option (blank or No). This prevents the guest VM from being exposed without a firewall protecting it.

Creating Shorewall rules

In the original guide all traffic from the public side to internal VMs is blocked. This continues to be the same when you are using proxy ARP and thus need to create rules that permit traffic to certain ports. This can be done by adding rules like:

ACCEPT    net    dmz:92.22.33.44    udp    5060

This will permit UDP traffic on port 5060 to proceed to the guest VM on IP 92.22.33.44. You can also use the Shorewall macros, for example:

HTTP/ACCEPT    net     dmz:92.22.33.44

And in this case it will make a web server on 92.22.33.44 accessible to the public.

Alternatively, you may setup a single rule that accepts all traffic on all ports:

ACCEPT    net     dmz:92.22.33.44

However you should take care that the guest VM has its own set of rules to block unwanted or unsafe traffic.

The main thing to take care of with these proxy ARP firewall rules is to use “ACCEPT” and not “DNAT” as explained in the original guide.

Multiple public IPs

Another common question is in regards to multiple public IP addresses. The original guide assumed that the host has one public IP address, so here are a few additional pointers.

If you are following the original guide, then any public IP will be applied the same rule. So let’s say you have two public IP addresses, 94.11.22.33 and 94.22.33.44 and the following rule:

ACCEPT    net    fw    tcp    5900:5999

With this rule, you can use both 93.11.22.33:5900 or 94.22.33.44:5900 to connect to Proxmox’s VNC. But you wish to restrict it to a specific IP address, then you need to modify the rule as following:

ACCEPT    net   fw:94.22.33.44    tcp    5900:5999

Now you can only use 94.22.33.44:5900 to connect, but not 93.11.22.33:5900.

The same applies to port forwarding (DNAT) rules, which is written in a slightly different format. Let’s say we have this rule:

DNAT    net    dmz:10.0.0.1    tcp    1234

This will forward any public IP to port 1234 on the guest VM at 10.0.0.1. So both 94.11.22.33:1234 and 94.22.33.44:1234 will work. If you wish to restrict this to a certain public IP address, the rule needs to be modified as following:

DNAT    net    dmz:10.0.0.1    tcp    1234    -    94.22.33.44

Only 94.22.33.44:1234 will be allowed at this point.

Memorable names

If you have many IP addresses, it becomes easy to forget which IP to use. You can use Shorewall’s “params” file to give IPs a memorable name. Edit the file /etc/shorewall/params as following:

IP_ALEX=94.11.22.33

IP_SONIA=94.22.33.44
IP_ERIC=94.33.44.55
IP_VM1=10.0.0.1
IP_VM2=10.0.0.2
IP_VM3=10.0.0.3

Now you can use these names instead of the IP address in any of your Shorwall rules, like so:

HTTP/DNAT    net     dmz:$IP_VM1    -    -    -    $IP_SONIA

And this would forward HTTP traffic from 94.22.33.44 to a guest VM running on 10.0.0.1.

Mixed use and bridging

The original guide not only adds a firewall, but also helped those who’s hosting provided blocked IPs on unauthorized MACs. One such hosting provided was OVH, and they have recently introduced a “Virtual MACs for VPS” option that allows you to assign an IP address to a MAC. This eliminates the need for Proxy ARP or port forwarding/NAT, however the original guide will still prove itself useful for protecting the host.

I will provide a sample setup below, which can be modified to suit your needs and network configuration. This assumes you have have a basic understanding of networking and read my previous guide.

Interfaces

We modify the /etc/network/interfaces as following:

# The loopback network interface
auto lo
iface lo inet loopback
# Public Network. Make sure to only use MACs that were assigned to you
auto vmbr0
iface vmbr0 inet static
# The following settings are specific to your hosting provider:
address 94.11.22.33
netmask 255.255.255.0
network 94.11.22.0
broadcast 94.11.22.255
gateway 94.11.22.254
# The following assumes eth0 is the public-side NIC, the remained is always the same
bridge_ports eth0
bridge_stp off
bridge_fd
# Optional Private Network. This network cannot be access directly from the public side
auto vmbr1
iface vmbr1 inet auto
address 10.0.0.1
network 255.0.0.0
broadcast 10.255.255.25
bridge_ports none
bridge_stp off
bridge_fd 0
The host, accessible by 94.11.22.33, will have two bridges at this point: vmbr0 serving the public-side and vmbr1 serving inter-VM communication.
To assign a public IP address to a KVM, you use the vmbr0 bride and must ensure the MAC corresponds to the one provided by your hosting provider (if a MAC restriction is in place).You can also add a 2nd NIC (or as the only NIC) to the KVM, which is bridged with vmbr1. In this case, you need to use an IP within the 10.0.0.2-10.255.255.254 range, and as gateway 10.0.0.1. This particular IP range can only be used between other VMs on the same host / bridge, unless you use port forwarding or a VPN on the same IP range.Shorewall Interfaces and ZonesThe basic /etc/shorewall/interfaces will be:#ZONE INTERFACE BROADCAST OPTIONS
pub vmbr0 detect routeback,bridge
loc vmbr1 detect routeback,bridge
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE
You could also add the blacklist option, or any of the other possible Shorewall options if you wish.
The accompanying /etc/shorewall/zones will look like this:#ZONE TYPE OPTIONS IN OUT
# OPTIONS OPTIONS
fw firewall
pub ipv4
loc ipv4
#LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE
You now have three distinct Shorewall zones. The fw zone is a self-reference to the host (in our sample setup 94.11.22.33 and 10.0.0.1). The pub zone represents the publicly accessible vmbr0 bridge and loc our internal vmbr1 bridge.
Shorewall PolicyThe following policy defines these basic rules:
  • Traffic from the the host anywhere else is permitted
  • Traffic from the public side to the host and the internal network is denied
  • Traffic from the internal side to the host is denied, anywhere else is permitted
Edit /etc/shorewall/policy:#SOURCE    DEST    POLICY     LOG      LIMIT:       CONNLIMIT:
# LEVEL BURST MASK
# From Firewall:
fw fw ACCEPT
fw pub ACCEPT
fw loc ACCEPT
# Public Bridge (read the policy warnings!):
pub pub ACCEPT
pub loc ACCEPT
pub fw DROP info
# Local (internal) Bridge:
loc loc ACCEPT
loc pub ACCEPT
loc fw DROP info
# THE FOLLOWING POLICY MUST BE LAST
#
all all REJECT info
#LAST LINE -- DO NOT REMOVE

Policy Warnings

With this sample policy it means that each publicly accessible guest VM should have its own firewall. If you wish to change this behavior, and let Shorewall handle the firewall for each such guest VM, then change the Public Bridge section:

pub pub DROP info

You then need to create specific rules that allow traffic to VMs on vmbr0.

A similar warning applies to traffic from vmbr0 to vmbr1. The policy assumes that vmbr0 does not receive any routable traffic on a private IP range (also called “martians”). Although this is often the case, it depends on the hosting provider’s internal networking. If you are not sure whether there’s routable traffic on a private IP range from the public side, you have two options. The first is to disallow all traffic from vmbr0 (pub) to vmbr1 (loc) by editing the Public Bridge section:

pub loc DROP info

The alternative is to have a set of rules in /etc/shorewall/rules like so:

# …
SECTION NEW
# Leave these at the top, right after “SECTION NEW”!
DROP pub:10.0.0.0/8 all
DROP pub:192.168.0.0/16 all
DROP pub:172.168.0.0/12 all
# … Other rules follow …
#LAST LINE — ADD YOUR ENTRIES BEFORE THIS ONE — DO NOT REMOVE

WARNING! In all cases, you should verify whether traffic is truly blocked and never assume that it is!

Shorewall and Fail2ban

Fail2ban, in its own words, “scans log files like /var/log/pwdfail or /var/log/apache/error_log and bans IP that makes too many password failures. It updates firewall rules to reject the IP address”. This can for instance be used to (temporarily) ban a bot that is attempting a brute-force entry through SSH.

Fail2ban is an ideal companion to Shorewall and can be installed in a matter of minutes on a Proxmox host. You start by installing Fail2ban from the Debian packages:

apt-get install fail2ban

Configure Shorewall

Next you need to edit one line in the Shorewall configuration file, located at /etc/shorewall/shorewall.conf:

BLACKLISTNEWONLY=No

That’s all you need to configure in Shorewall. Remember to apply your settings by restarting Shorewall with the command “shorewall restart”.

Configure Fail2ban

The last step is to configure Fail2ban. The file /etc/fail2ban/jail.conf is extensively documented and of particular interest are the following settings


#
# Destination email address used solely for the interpolations in
# jail.{conf,local} configuration files.
destemail = myemail@address.com

#
# ACTIONS
#
# Default banning action (e.g. iptables, iptables-new,
# iptables-multiport, shorewall, etc) It is used to define
# action_* variables. Can be overriden globally or per
# section within jail.local file
banaction = shorewall

# Choose default action. To change, just override value of ‘action’ with the
# interpolation to the chosen action shortcut (e.g. action_mw, action_mwl, etc$
# globally (section [DEFAULT]) or per specific section
action = %(action_mwl)s

Inform fail2ban of these changes by issuing the following command:

fail2ban-client reload

The destemail variable should be changed to your own e-mail address, where you will be informed of any ban actions. The banaction variable specifies that Shorewall should be used to block possible intruders. And finally, the action variable tells fail2ban to ban any detected intruder and then send an you a detailed e-mail with the relevant log lines (that caused the ban).

Following is an example of an actual e-mail:

Hi,
The IP 193.86.5.103 has just been banned by Fail2Ban after
3 attempts against ssh.
Here are more information about 193.86.5.103:
% This is the RIPE Database query service.
% The objects are in RPSL format.
%
% The RIPE Database is subject to Terms and Conditions.
% See http://www.ripe.net/db/support/db-terms-conditions.pdf
% Note: This output has been filtered.
% To receive output for a database update, use the “-B” flag.
% Information related to ‘193.86.4.0–193.86.5.255’
inetnum: 193.86.4.0–193.86.5.255
netname: BRANO
descr: BRANO, Inc.
descr: Hradec nad Moravici
country: CZ
admin-c: BK230-RIPE
tech-c: TP231-RIPE
status: ASSIGNED PA
mnt-by: GTSCZ-MNT
source: RIPE # Filtered
person: Bohumil Kriz
address: BRANO, Inc.
address: Computer Centre
address: Hradec nad Moravici
address: 747 41
address: The Czech Republic
phone: +420 653 918118
fax-no: +420 653 911791
nic-hdl: BK230-RIPE
source: RIPE # Filtered
person: Tomas Partl
address: Brano, Inc.
address: Computer Centre
address: Hradec nad Moravici
address: 747 41
address: The Czech Republic
phone: +420 653 918371
fax-no: +420 653 911791
nic-hdl: TP231-RIPE
source: RIPE # Filtered
% Information related to ‘193.86.0.0/16AS2819’
route: 193.86.0.0/16
descr: CZNET-A
origin: AS2819
mnt-by: GTSCZ-A-MNT
source: RIPE # Filtered
Lines containing IP:193.86.5.103 in /var/log/auth.log
Feb 27 03:32:02 host sshd[21460]: Failed password for root from 193.86.5.103 port 35833 ssh2
Feb 27 03:32:04 host sshd[21502]: Failed password for root from 193.86.5.103 port 36302 ssh2
Feb 27 03:32:06 host sshd[21504]: Failed password for root from 193.86.5.103 port 36719 ssh2
Regards,
Fail2Ban

By default fail2ban is configured to monitor SSH, which for a Proxmox host works without any additional changes. Personally I prefer a ban on 3 attempts instead of the default 6, so I have made this one change in the /etc/fail2ban/jail.conf file:

[ssh]
maxretry = 3

But again, the configuration file is quite well documented, so any personal preferences or modifications should be easy to accomplish.

Myatu’s

Myatu’s Tech Blog, from the site that has been wasting bits and bytes daily, since 2008.

Mike Green

Written by

I keep servers happy, and they keep me happy.

Myatu’s

Myatu’s

Myatu’s Tech Blog, from the site that has been wasting bits and bytes daily, since 2008.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade