Minimal Cache Proxy: Configuring An HTTPS Everywhere Squid Plugin on OPNsense

4 min readAug 2, 2017

How to install a rule set based URI rewriting plugin to increase privacy

Tl;dr

The HTTPS Everywhere rule set, which contains web sites that can be bumped up from unencrypted HTTP to encrypted HTTPS, can be used by a web proxy running on OPNsense. By adding a specialist squid plugin there is a reduction in caching, but greater privacy and security for traffic passing to/from all clients that use the proxy.

From: https://en.wikipedia.org/wiki/File:Internet2.jpg

Introduction

A n OPNsense system is suitable for various network functions. One of these is the squid web caching proxy, which can operate transparently to handle HTTP traffic on TCP port 80. Whilst caching might be useful to some, the fundamental property that allows caching is unencrypted traffic. For privacy (and security) the minimisation of plaintext traffic should be a priority; reducing the potential for content monitoring and modification.

HTTPS Everywhere (from EFF and Tor) is a very worthy project that aims to increase the usage of HTTPS traffic by using a set of rules containing web sites that “offer some limited support for encryption over HTTPS” but “may default to unencrypted HTTP, or fill encrypted pages with links that go back to the unencrypted site”. It normally operates as a browser extension (aka addon) that uses the rules to rewrite URIs to use HTTPS.

However, this can also be carried out beyond the user agent using a web proxy on the network. This has the benefit of being independent of any clients, so all applications using HTTP can be bumped up to HTTPS — not just those with the browser extension installed. The downside is that the rule set might cause some breakages in some cases.

Squid can make use of plugins to add functionality, and Mike Cardwell wrote an HTTPSEverywhere Perl library and squid plugin (which is also used by Dowse). This has been adapted to work correctly and on OPNsense (which is a BSD variant). The guide below will show how to set it up.

Web Proxy

I t is assumed that OPNsense (v17.7 at the time of writing) is installed and working. If not already configured, squid needs to be up and running. As a point of note, assuming one is concerned with end-to-end privacy, TLS/SSL interception (inspection) using squid should not be enabled.

Transparent mode is very useful for most scenarios, though the corresponding ‘Firewall’ and ‘NAT — Port Forward’ rules need to allow/pass and redirect TCP port 80 to the proxy port (3128) on localhost. Redirect rules are also needed if the OPNsense system itself should use the proxy. Rules to deny/block attempted proxy bypass should be added, too.

Another potentially useful feature is filtering request URIs to prevent undesirable interaction. Third party lists include Shalla’s Blacklists and the Université Toulouse 1 Capitole Blacklists. These contain categories that can be enabled or disabled as wanted.

HTTPS Everywhere Plugin

Login to use a root shell — the web interface cannot be used to configure the rest. Install git (used for acquiring the scripts and rule set).

# pkg install git

The HTTPS Everywhere Perl script requires the LibXML library, which is not available from the OPNsense repository. Therefore, the FreeBSD repo can be enabled to install the dependencies.

/usr/local/etc/pkg/repos/FreeBSD.conf

FreeBSD: { enabled: yes }

Update the package cache, then install Perl’s LibXML

# pkg update
# pkg install p5-XML-LibXML

Next create the plugin directory, then fetch the plugin and assistance files. Always check any software that has been downloaded from the Internet — it might be malicious.

# mkdir -v -p /usr/local/libexec/squid/https-everywhere
# git clone 'https://gitlab.com/privb0x23/perl-https-everywhere' /usr/local/libexec/squid/https-everywhere
# cd /usr/local/libexec/squid/https-everywhere

The plugin script (squid.pl) needs to be executable by the squid user. The rule set update shell script can be configured to be run periodically by cron.

# mkdir -v git
# chmod 755 squid.pl update_rules.sh
# ./update_rules.sh

Squid’s configuration needs to include the plugin script. Create a new file that’s automatically referenced by the main config file. Adjust as necessary.

/usr/local/etc/squid/post-auth/https-everywhere.conf

redirect_program /usr/local/libexec/squid/https-everywhere/squid.pl
url_rewrite_children 5

Finally, using the web interface, restart the squid service. Test using a browser that does not have the HTTPS Everywhere extension installed, or disable it during testing. For example, try ‘http://www.irongeek.com/’.

Rules

The update_rules.sh script will update the rule set, which will be automatically absorbed by the squid plugin every hour (though a manual restart of the squid daemon will also re-read the rules).

Custom rules can be added, if desired.

/usr/local/libexec/squid/https-everywhere/git/https-everywhere/src/chrome/content/rules/custom_rule.xml

<ruleset name="Custom Rule">
  <target host="noiseprotocol.org" />
  <target host="*.noiseprotocol.org" />
  <rule from="^http:" to="https:" />
</ruleset>

Evaluation

The squid plugin adds the HTTPS Everywhere functionality and works to increase the use of encrypted traffic for all clients using the web proxy. The performance implications haven not been investigated, but there is some additional processing for each HTTP request on TCP port 80. Some web sites might cause problems, but those rules that are deemed problematic (‘default off’ or ‘mixed content’) are not enabled.