Network environments are growing increasingly hostile. Freedoms of speech, religion, and press are being diminished. The right to privacy is challenged globally on multiple fronts. For these and other reasons, you may find it valuable to add an extra layer of security and privacy to your Internet connection.
This is typically done using VPNs, SSH tunnels, or just regular ol’ HTTP proxies. However, they each have their problems:
- VPN: When used primarily for anonymized/proxied browsing, VPNs are less than ideal. VPN traffic is fairly obvious on the wire, especially with deep packet inspection, which is why governments are able to block (most) VPNs if they try to. VPN providers often advertise anonymity and protection while running sketchy operations that log your activity or don’t properly secure your traffic. To make matters worse, setting up your own VPN correctly can be difficult. Conflicting, incompatible protocols sometimes require installing extra software depending on the VPN you connect to.
- SSH tunnel: SSH is available on most Linux machines (and Macs), but like VPNs, SSH traffic is pretty easy to identify — mostly because it runs on port 22. Running services on a different port is not always an option because of firewall restrictions. Even if you multiplex multiple protocols on the same port, SSH traffic looks different from HTTPS. Also, SSH is known for being unreliable when the network drops out (“broken pipe” anyone?), and slow every other time. But at least SSH tunnels are much simpler to create than setting up a VPN.
- HTTP proxy: The obvious requirement here is for HTTPS at the very least, but unless you properly automate a Let’s Encrypt certificate, you have to go to some work to maintain the HTTPS server. Pretty much all HTTP proxies these days do not support HTTP/2, and it’s pretty easy to tell (from the client-side) if a server is acting as a forward proxy. I should not even have to mention that trusting random HTTP proxies on the Internet isn’t a great idea.
This article will show you how to secure your Internet connection in a way that overcomes these weaknesses. You will learn how to set up the Caddy web server as a secure forward proxy using the http.forwardproxy plugin, which has these advantages:
- HTTPS enabled by default. All certificate maintenance is automated. It just works!
- Full-duplex HTTP/2. This proxy speaks HTTP/2 both ways, which gives you faster page load times.
- End-to-end encryption. For sites that use HTTPS, your connection to the origin is E2E encrypted. Even the proxy server cannot decrypt your connection between your computer and the origin site. Once the HTTPS tunnel is established with the proxy, it simply shuttles bytes in a two-way stream. Underlying TLS connections remain intact. (Even a nefarious proxy can’t read the content of your underlying HTTPS traffic, but you should still only use proxies you trust. There’s more to traffic than its contents.)
- Probe resistance (experimental). This feature hides the proxy behavior of the server from everyone except to those who already know how to access it and are authorized to use it. Only a secret link specified by you (the server operator) exposes the authentication prompt.
- Innocuous traffic patterns. Since this proxy is not a VPN or SSH tunnel, it blends in better with the rest of HTTPS traffic.
- Standard ports. Where VPN ports and SSH port 22 are blocked by firewalls, port 443 is almost always allowed because accessing the Web is so standard for most use cases. (But you can still customize the port.)
- Hides your IP address. With just one line of config, this proxy will not add the user’s IP to the HTTP “Forwarded” header. (Note that there are ways around this out of the proxy’s control, like WebRTC in browsers.)
- Access controls. You can specify users and passwords to restrict access to the proxy, as well as a port whitelist for the proxy.
- Easy to set up! Best of all, this proxy is easy to get running and hard to get wrong because of sane defaults and automatic HTTPS. Caddy is designed to be easy to use to reduce the potential error surface of misconfigurations.
In particular, we’re interested in exploring how helpful this software software could be in circumventing censorship. It definitely needs more careful vetting (read on) but we hope people will try it out in low-or-no-risk scenarios.
This plugin was developed by Sergey Frolov while interning at Google and the source is available on GitHub.
Presented “As-Is”
I make no guarantees; use this tutorial and the server software at your own risk. There are a number of edge cases that ultimately depend on your client and your threat model; see especially the warning in the client configuration section below.
This technique is still fairly new. We want to make it even safer. So we do not recommend using it in high-risk situations. If you find a way to improve it, though, please contribute your feedback, issues, and pull requests!
Setting up the Server
First, you’ll need a machine that is accessible with a public IP address. Home servers can work if you forward the port(s) properly. You can also rent a cloud instance from any reputable cloud provider for a few dollars per month. Once you have such a machine, here’s what to do:
Download Caddy with the http.forwardproxy plugin included. To do that, make sure you select it in the plugins list on the download page!
Install Caddy; this is as easy as extracting the archive and putting the binary in your PATH, or using the one-line auto-installer script shown at the bottom of the download page after you select the plugins you want.
Make a file called Caddyfile that looks like this:
example.comroot /path/to/your/site
forwardproxy {
basicauth user pass
probe_resistance secret.localhost
hide_ip
}
You must replace:
example.comwith the actual domain name pointed at your machine/path/to/your/sitewith the actual path to the root of your site (or an empty or decoy folder if you have no site)user passwith a username and password of your choice (otherwise anyone could use your server!)secret.localhostwith a custom, secret hostname to enable probe resistance; strongly recommended to end with “.localhost”
Then run caddy in the same folder as your Caddyfile. After a few seconds, your probe-resistant, IP-hiding, full-duplex, HTTP/2 proxy will be running with a fully-managed TLS certificate from Let’s Encrypt!
The default port is 443 (the HTTPS port) unless you specify otherwise. Read more about the Caddyfile if you want to customize further.
Setting up the Client
Now how do you use your proxy server? With a client, of course!
There are several ways to do this. For example, if you want to secure your web browsing in Chrome, you can use an extension like Proxy SwitchyOmega to configure the connection. Firefox has some network options built into its settings. You can also configure your entire OS or mobile phone to use the proxy for all applications (except those that are specifically configured to not use your OS’ proxy settings). However, we found only Chrome+SwitchyOmega to be reliable, which we explain below.
** WARNING! A weakness in any part of the proxy configuration could leak information. Even if your proxy server is secure, your client may not be. Clients which do not honor the proxy settings for all network traffic could put you at risk. This includes major browsers and operating systems. For example, browsers don’t put WebRTC requests through the proxy (by design, sigh). One way to mitigate this risk somewhat is to use a VM that tunnels all traffic across a virtual NIC. This is obviously more involved, so act according to your threat model. If you just need basic public Internet cafe privacy to check your email, maybe the VM is overkill. That’s up to you.
Chrome + SwitchyOmega
This was the most reliable and simple client configuration I tried.
Install SwitchyOmega. It comes with an example proxy profile (on the left) which you can modify, or you can create a new one. In the table, select “HTTPS” for the “Protocol” field and type in your domain name and the port:
Click the lock icon by the port and enter your username and password:
After save your credentials, click the green “Apply Changes” button to the left:
You can turn on the proxy by clicking it from your menu:
Congrats! Now all your Chrome connections are proxied securely through your server.
Firefox
Like Chrome, Firefox also has a SwitchyOmega extension. I would suggest using extension first, since you are likely to have more success than with Firefox’s built-in network settings. The instructions are nearly the same as above.
Using Firefox’s built-in network configuration settings, I was not able to get this working in Firefox (on macOS 10.12.6) for a server with probe_resistance enabled. It appears to be a bug in Firefox. Firefox hangs when loading the page, for several minutes — and spins the CPU at 100%. Even after closing the application, my computer ran sluggish for almost a day until I found the firefox process still running in the background still using 100% of my CPU. Other than using the SwitchyOmega extension, there are two other workarounds on Mac: Use Chrome, or disable probe resistance.
To change Firefox’s built-in network configuration, go to Menu -> Preferences:
Then Network Proxy, and click “Settings…”:
Here you have a few options. You can “Use system proxy settings” which should use your operating system’s proxy settings. You can specify a “Manual proxy configuration” where you enter the hostname and port your proxy is listening on (EDIT: As noted in a comment, this will not utilize TLS-to-proxy. So don’t do this, use the PAC file instead — next sentence). Or, if you use the serve_pac server setting, you can specify its URL:
Save your settings, and you should be good to go. Maybe. Good luck!
macOS (and/or Safari)
This will change the proxy settings for the entire system. Make sure this is really what you want to do instead of using the proxy only with your web browser!
Unfortunately, I was not able to successfully get macOS’ system-wide proxy configurations to work reliably, even with probe resistance / authentication turned off. I’ve also been told that some previous version(s) of macOS (or OS X) could even kernel panic when using a TLS proxy. (But I was able to get Firefox and Chrome using the proxy successfully, as described above.)
If you want to experiment with getting system-wide proxy config working, open System Preferences and go to Network. Choose your active network interface on the left and click the “Advanced” button in the lower-right. Click the “Proxies” tab. I tried both the “Automatic Proxy Configuration” (where you give the .pac file URL) and the “Secure Web Proxy (HTTPS)” options. Both had different but equally disappointing levels of support in various applications.
Linux (Ubuntu 16.04 Desktop)
Good news! Setting Linux’s system-wide proxy settings worked fairly well in my testing.
Open Network preferences and select “Network Proxy”. Choose “Automatic” for the Method, and type the configuration URL of the .pac file. (You’ll have to enable serve_pac on your server configuration inside the forwardproxy directive; choose a secret URL to serve the file on, so it doesn’t defeat your probe resistance). If you don’t want to use a .pac file, you can choose “Manual” for the Method and enter your hostname and port (443 probably) — use it for all protocols.
Since there isn’t a way to configure the system proxy to send credentials, you need to use your secret link to authenticate. Only the secret link will prompt the browser to enter credentials. You may have to do that every time you re-open your browser.
So open your web browser (maybe double-check its network configuration if you’re not sure) and go to your secret link. Enter your credentials, and you’re good to go!
Android
This method works for WiFi networks. I haven’t looked into how to set a proxy for the cell network. It requires using a .pac file. If you haven’t already, enable the serve_pac option within the forwardproxy directive. For probe resistance, I recommend specifying a secret URL to serve it on, rather than the default /proxy.pac.
Go to your WiFi settings and long-tap on the current network. Choose “Modify Network”:
Under Proxy, choose “Proxy Auto-Config” and type the URL to your .pac file:
Save settings. With probe resistance enabled, you’ll have to navigate to your secret link in your browser to expose the authentication prompt. Then you should be good to go. Remember: this doesn’t apply to the cell network.
Windows
**IMPORTANT NOTE** While this can work on Windows, apparently Windows does not support TLS-to-proxy, meaning your transmissions to your proxy server will not be properly secured. If you want this fixed, you should raise a stink with Microsoft. I don’t have a Windows machine to confirm this, so in the meantime, use another solution.
These instructions are for Windows 7, but the same basic idea works in more recent versions of Windows. I’m borrowing these instructions from Sergey’s blog (with permission) because I don’t have Windows:
Control Panel → Network and Internet → Internet Options → Connections → LAN settings → Check “Use a proxy server…” and paste your “https://yourserver.com” in Address and “443” in port. Don’t lose “https://” in Address, Windows likes to remove it when you open LAN settings window again.
Conclusion
Caddy’s http.forwardproxy plugin is a promising alternative to using VPNs or SSH tunnels for certain tasks and for certain threat models. We hope this technology will help provide greater access to the Web to more people because of its unique properties and ease of use. However, client support for TLS-to-proxy must improve and become more reliable and predictable. We hope that client support for secure proxying will improve and become more robust in the future.
