A practical guide to containers on FreeNAS for a depraved psychopath.

This is a simple write-up to setup Docker on FreeNAS 11 or FreeBSD 11.

But muh jails?

You know that jails are dope and you know that jails are dope, yet no one else knows it. So here we are stuck with docker. Two years ago I would be the last person to recommend using docker, but a whole lot of things has changes past years…

So jails are dead then?

No, jails are still dope, but jails lack tools to manage them. Yes, there are a few tools, but they meant for hard-core FreeBSD users who used to suffering. Docker allows you to run applications without deep knowledge of application you’re running. It will also allow you to run applications that are not ported to FreeBSD.

Prerequisites

All you need is terminal with root access via sudo.

(Optional) FreeNAS setup

You might think that VM tab in FreeNAS 11 is your friend… Well, it’s not. It’s only your friend when your guest can boot on UEFI system. This feature is not complete in FreeNAS…yet…

Enter BHyve

In case you don’t know FreeBSD now has its own hypervisor and it even can run Windows! No joke. Just like anything else in FreeBSD it’s more of a framework rather than a complete product. Likely someone built tools to simplify it — iohyve and chyves. This guide will follow iohyve way because I forgot about chyves and whole drama around this.

iohyve vs chyves

tl;dr chyves is a fork of iohyve because the author of iohyve sometimes is hard to work with. If your goal is to run docker you don’t care about the iohyve issues. if you choose to use chyves, then translate commands to chyves, it ain’t hard.

(Optional) Setup iohyve

If you’ve never used iohyve before do this:

iohyve setup pool=zpool kmod=1 net=em0

Where zpool is your pool name and em0 your network card. This will create iohyve zfs dataset.

Install docker host OS

There are many variants you can choose from — RancherOS, CoreOS are the most popular for docker-only hosts. We going to use RancherOS because it’s more lightweight of the box.

Download installer

Navigate to RancherOS website and grab link to latest. Right now it’s v1.0.4 — Docker 17.03.1-ce — Linux 4.9.40

sudo iohyve fetch https://releases.rancher.com/os/latest/rancheros.iso
sudo iohyve renameiso rancheros.iso rancheros-v1.0.4.iso

Create VM

Before you create host you going to need to install sysutils/grub2-bhyve port. As usual, you can do it with pkg install grub2-bhyve

sudo iohyve create rancher 32G # Substitute 32 with whatever you want.
# You can always add more storage later.
sudo iohyve set rancher loader=grub-bhyve ram=8G cpu=8 con=nmdm0 os=debian
# Again, change ram and cpu values if you need to.
sudo iohyve install rancher rancheros-v1.0.4.iso

In a separate terminal:

sudo iohyve console rancher
# to exit console type ~~. really quick.
# now you going to see grub console. Hit enter once if you don't believe me.
set root=(cd0,msdos1)
linux /boot/vmlinuz* ro rancher.password=rancher
initrd /boot/initrd*
boot
# Wait for OS to boot. 
# Password to rancher user is rancher in case you don't read what you pasting into things ran as root.

Create cloud-config.yml

#cloud-config
ssh_authorized_keys:
  - ssh-rsa YOUR-SSH-KEY

#/var/lib/rancher/conf/cloud-config.d/netconfig-config.yml
rancher:
  network:
    interfaces:
      eth0:
        addresses:
          - 192.168.86.200/24
        gateway: 192.168.86.1
        dhcp: false
    dns:
      nameservers:
        - 8.8.8.8
        - 8.8.4.4

Change eth0settings to what’s suits you.

sudo ros config validate -i cloud-config.yml
# should show no errors
sudo ros install -c cloud-config.yml -d /dev/sda

Installer will ask you if you want to reboot — say yes. It won’t reboot…

Back to FreeBSD

Create file /mnt/iohyve/rancher/grub.cfg:

set root=(hd0,1) 
linux /boot/vmlinuz* printk.devkmsg=on rancher.state.dev=LABEL=RANCHER_STATE rancher.state.wait console=tty0 ro rancher.password=<YOUR PASSWORD OF CHOICE>
initrd /boot/initrd*
boot

Run this on your host (FreeBSD):

sudo iohyve set rancher os=custom
sudo iohyve set rancher boot=1
sudo iohyve start rancher

After some time you should be able to ssh into the machine. Add those to /etc/rc.conf:

ng_ther_load = YES
iohyve_flags = kmod=1 net=em0
iohyve_enable = YES

(Optional) Install Portainer

Whole point of this guide is to reduce pain, and using docker CLI is still painful. There are a lot of Web UI to control docker. Most of them include a lot of orchestrating services, so it’s just overkill. Portainer is very lightweight and can be run even on Raspberry Pi.

Create /var/lib/rancher/conf/cloud-config.d/portainer.yml:

#/var/lib/rancher/conf/cloud-config.d/portainer.yml
rancher:
  services:
    portainer:
      image: portainer/portainer
      container_name: "portainer"
      environment:
        PGID: '1101'
        PUID: '1101'
      labels:
        io.rancher.os.after: console, preload-user-images
      privileged: true
      restart: always
      ports:
        - "9000:9000"
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock
        - /mnt/config/portainer:/data

After reboot you will be able to access WebUI on 9000 port. Setup is very easy, so I won’t go over it.

(Optional) Enable Remote API

FreeBSD can’t host containers itself, but it can manipulate remote docker hosts. Install sysutils/docker and follow this instructions.

(Optional) Enable TLS-Less Remote API

No.

(Optional) Enable NFS

Don’t. It might look like a good idea, but it isn’t. Proper way would be the way FreeNAS Coral does it: p9

(Optional) Set timezone in container

Don’t. Trust me. Use UTC everywhere. Timezone is a matter of UI and UI should take care of presenting dates in a viewer timezone.