Set up your own home server lab

*This is an old post from when I had my servers hosted on my own machines. I now host the below sites on the cloud.*

Like wasting food, throwing away a usable machine always irks me a little. So rather than pay for hosting in the cloud, I’ll take old machines and build server laboratories out of them so I can play with different stacks and frameworks. This becomes especially easy with hypervisors and docker! Add LetsEncrypt, and I get free SSL too!

Since this is my first post, I’ll give a little background about myself. I come from (mostly) a Microsoft background and I recognize there are many things that Microsoft does well and many that it doesn’t. However, with Satya Nadella’s direction of “Microsoft loves Linux” and the company’s embracing of open-source, we now have the ability to make the best of all worlds!

In this article, I will discuss how I took a desktop computer and turned it into a mini-server farm hosting Windows and Linux applications all working seamlessly behind one dynamic IP with a properly signed SSL certificate.

Setup

I used a few desktops with ESXI 6.5 Hypervisor. I also have a few domains I own. Below is my high level network diagram. I will discuss the details later in this post.

DNS and SSL

Since I host my servers at home behind a public FIOS IP and I refuse to pay Verizon a bunch of money for a static one, I use a dynamic DNS provider. I use [subdomain].crabdance.com. I then bought two domain names (saninsoftware.com and saninstravels.com) and created a CNAME record for [subdomain].crabdance.com. Now, whenever my IP changes, my dynamic DNS is updated automatically and my other domains know where to go!

For SSL, I use LetsEncrypt. It’s a tool that gives you a 90 day SSL for free! Most modern browsers honor LetsEncrypt certs so you won’t get those annoying “Could not verify this site” browser warnings. The catch is, that every 90 days you have to renew the cert but thanks to the LetsEncrypt tools, this is quite easy.

Server Components

CentOS with HAProxy

I created a standalone VM with CentOS and HAProxy only. Originally I configured HAProxy to do SSL passthrough via SNI but it became complicated when it handled multiple domains. So to keep it simple, I decided it would handle all SSL and therefore it became a dedicated VM. This could cause horizontal scale-ability issues later, but that’s a problem for another day.

My HAProxy config looks like this:

global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
user foo
group foo
daemon
maxconn 2048
tune.ssl.default-dh-param 2048
defaults
log global
mode http
option httplog
option dontlognull
option forwardfor
option http-server-close
timeout connect 5000
timeout client 50000
timeout server 50000
stats enable
stats auth fakeuser:fakepassword
stats uri /stats
frontend http-in
bind *:80
# Define hosts based on domain names
acl host_saninsoftware hdr(host) -i www.saninsoftware.com
acl host_saninstravels hdr(host) -i www.saninstravels.com
acl host_websql hdr(host) -i websql.saninsoftware.com
acl host_stories hdr(host) -i stories.saninsoftware.com
acl host_ssblog hdr(host) -i blog.saninsoftware.com

redirect scheme https if host_websql
        ## figure out backend to use based on domainname
use_backend saninsoftware if host_saninsoftware
use_backend saninstravels if host_saninstravels
use_backend websql if host_websql
use_backend stories if host_stories
use_backend ssblog if host_ssblog
frontend https-in
bind *:443 ssl crt /path/to/sslkey.pem

reqadd X-Forwarded-Proto:\ https
# Define hosts based on domain names
acl host_saninsoftware hdr(host) -i www.saninsoftware.com
acl host_saninstravels hdr(host) -i www.saninstravels.com
acl host_websql hdr(host) -i websql.saninsoftware.com
acl host_stories hdr(host) -i stories.saninsoftware.com
acl host_ssblog hdr(host) -i blog.saninsoftware.com
        ## figure out backend to use based on domainname
use_backend saninsoftware if host_saninsoftware
use_backend saninstravels if host_saninstravels
use_backend websql if host_websql
use_backend stories if host_stories
use_backend ssblog if host_ssblog
backend saninsoftware # www.saninsoftware.com container (.NET Core)
balance roundrobin
option httpclose
option forwardfor
server saninsoftwarecontainer 192.168.2.3:1080

backend websql # websql container (.NET Core)
balance roundrobin
option httpclose
option forwardfor
server websqlcontainer 192.168.2.3:1081

backend stories # stories container (.NET 4.5)
balance roundrobin
option httpclose
option forwardfor
server storiescontainer 192.168.2.3:1083
backend saninstravels # www.saninstravels.com container (WordPress)
balance roundrobin
option httpclose
option forwardfor
server saninstravelscontainer 192.168.2.6:2000
backend ssblog # This blog (WordPress)
balance roundrobin
option httpclose
option forwardfor
server ssblogcontainer 192.168.2.6:2001

Since there are some solid tutorials out there about HAProxy, I’m not going to explain my configuration line by line. However, the important thing to note is that all my domains use one SSL certificate.

Obtaining an SSL Certificate

I used a tool called Certbot which generates LetsEncrypt certificates automatically. One of the limitations, at the time of this writing, is that the certificates expire after 90 days. Many sites describe how to automate the certificate renewal via CRON jobs and that will be something I will explore so I don’t have to manually do it every 3 months. In addition, Certbot allows you to specify multiple domains per certificate so I didn’t have to tell HAProxy to look at multiple files.

MariaDB and SQL Server

Since databases should not be hosted in containers, I installed them on individual virtual servers using their installation guidelines.

Migrating Existing SaninsTravels WordPress Site

Since my previous blog (www.saninstravels.com) already existed, I wanted to determine how to migrate that to a docker container. I basically followed the guidelines established here. Since I had a lot of customizations, I copied my wp-content folder to /var/wp-content and mapped it in the new docker container. The docker-compose file looks like this:

version: '3.3'
services:
wordpress:
container_name: saninstravels-wordpress
build:
context: .
dockerfile: Dockerfile
ports:
- "2000:80"
volumes:
- "/var/wp-content:/var/www/html/wp-content"
restart: always
networks:
- wp-net
networks:
wp-net:
external: true

The docker file itself looks like this:

FROM wordpress
ADD wordpress /var/www/html
RUN usermod -u 1000 www-data
RUN usermod -G staff www-data
RUN chown -R www-data /var/www/html

Creating new WordPress site

This blog was a new WordPress site. I used this guide as a reference for my setup, though it’s just as good for other kinds of fresh Docker WordPress setups. My Docker compose file looks about the same as above:

version: '3.3'
services:
wordpress:
container_name: ssblog-wordpress
build:
context: .
dockerfile: Dockerfile
ports:
- "2001:80"
volumes:
- "/var/wp-content-ssblog:/var/www/html/wp-content"
restart: always
networks:
- ssblogwp-net
networks:
ssblogwp-net:
external: true

My docker file looks similar to the above also:

FROM wordpress:4.9.1-apache
ADD wordpress/wp-config.php /var/www/html/wp-config.php
RUN usermod -u 1000 www-data
RUN usermod -G staff www-data
RUN chown -R www-data /var/www/html

However, I had some permission issues when running this Docker file and created a bash script to build and run the image every time:

echo "starting"
docker-compose build --no-cache
docker-compose up -d
echo "sleeping..."
sleep 20s
docker exec ssblog-wordpress chown -R www-data /var/www/html

You may wonder why I needed the “docker exec” command at the end and my theory is that I needed to run “chown” after the container started up.

Dockerizing .NET 4.5 App With IIS SSL Dependency

I had an existing .NET 4.5 App (called Stories) which I wanted to Dockerize. The code had a dependency where it checked if it was being run under SSL in code. To get this to work in Docker, I first needed to set up Windows Server 2016. I then put all my binaries under a directory “/Stories”. I created a PowerShell script called Startup.ps1:

import-module webadministration
#Get-Module -ListAvailable
cd cert:
$cert = New-SelfSignedCertificate -DnsName myweb -Friendlyname MyCert -CertStoreLocation Cert:\LocalMachine\My
$rootStore = New-Object System.Security.Cryptography.X509Certificates.X509Store -ArgumentList Root, LocalMachine
$rootStore.Open("MaxAllowed")
$rootStore.Add($cert)
$rootStore.Close()
cd iis:
new-item -path IIS:\SslBindings\0.0.0.0!443 -value $cert
New-WebBinding -Name "Default Web Site" -IP "*" -Port 443 -Protocol https
iisreset

I then created a Docker file:

# The `FROM` instruction specifies the base image. You are
# extending the `microsoft/aspnet` image.
FROM microsoft/aspnet
# The final instruction copies the site you published earlier into the container.
COPY ./Stories/ /inetpub/wwwroot
COPY ./startup.ps1 /startup.ps1
RUN "powershell.exe c:\startup.ps1"

And then finally I executed my run with a PowerShell script like so:

docker rm -f storiesinstance
docker build -t stories .
docker run -d --name storiesinstance -p 1083:80 -p 1084:443 --restart always stories

Next Steps

Kubernetes! I want to replace all this orchestration with Kubernetes and utilize Infrastructure As Code (via Terraform) to get all these running.