Setting Up a Highly Available Kubernetus Cluster (K3S) on Hetzner Cloud with Terraform

Kubernetes and setup awesome: Longhorn, Traefik

5 min readFeb 1, 2024

Introduction:

Kubernetes is an excellent platform for orchestrating containerized applications. This article will guide you through the process of setting up a highly available Kubernetes cluster on Hetzner Cloud using Terraform, along with additional components like Longhorn and Traefik.

Getting Started:

Hetzner API Token

Create a project in your Hetzner Cloud Console, and go to Security > API Tokens of that project to grab the API key, it needs to be Read & Write. Take note of the key! ✅

Hetzner Cloud Console — Generate New Api Token

Generate SSH Key

Generate a passphrase-less ed25519 SSH key pair for your cluster; take note of the respective paths of your private and public keys.

ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_terraform_hetzner_cloudb

Create project folder and initialize snapshots and basic required files

Now navigate to where you want to have your project live and execute the following command, which will help you get started with a new folder along with the required files, and will propose you to create a needed MicroOS snapshot. ✅

mkdir my-kubernetes-project && cd my-kubernetes-project
tmp_script=$(mktemp) && curl -sSL -o "${tmp_script}" https://raw.githubusercontent.com/kube-hetzner/terraform-hcloud-kube-hetzner/master/scripts/create.sh && chmod +x "${tmp_script}" && "${tmp_script}" && rm "${tmp_script}"

once you run the above script, it will show you 4 steps prompts, we are going to fill every step in:

It will ask you about folder name, so you just need to hit Enter key
At this step it wants to create two snapshot image inside the hetzner cloud. So you must write yes to agree with.
Copy you Hetzer Api Token which you generated at the first and hit Enter key to continue.
After a several minutes the generated process will be finished and will be prompted you the snapshots’ ids. we need these two ids so make a note them.
==> Builds finished. The artifacts of successful builds are:
→ hcloud.microos-x86-snapshot: A snapshot was created: ‘OpenSUSE MicroOS x86 by Kube-Hetzner’ (ID: xxxxxxxx)
→ hcloud.microos-arm-snapshot: A snapshot was created: ‘OpenSUSE MicroOS ARM by Kube-Hetzner’ (ID: yyyyyyyyy)

Usage

Setup terraform environment

Create terraform.tfvars file and fill in the hcloud_token and snapshot_id image ids

hcloud_token                 = "hcloud_token obtained in the first step"
microos_x86_snapshot_id      = "xxxxxxxxx"
microos_arm_snapshot_id      = "yyyyyyyyy"

Create terraform provider file

Create Terraform Provider File (provider.tf):

terraform {
  required_version = ">= 1.5.0"
  required_providers {
    hcloud = {
      source  = "hetznercloud/hcloud"
      version = "1.45.0"
    }
    helm = {
      source  = "hashicorp/helm"
      version = ">= 2.0.1"
    }
  }
}

# Hetzner Cloud Provider
provider "hcloud" {
  token = var.hcloud_token
}

# Helm Provider
provider "helm" {
  kubernetes {
    config_path = "./k3s_kubeconfig.yaml"
  }
}

Edit terraform kube.tf file

Replace the content of kube.tf with the following module configuration:

module "kube-hetzner" {
  providers = {
    hcloud = hcloud
  }

  hcloud_token = var.hcloud_token

  source = "kube-hetzner/kube-hetzner/hcloud"

  ssh_port        = 2220
  ssh_public_key  = file("~/.ssh/id_ed25519_terraform_hetzner_cloud.pub")
  ssh_private_key = file("~/.ssh/id_ed25519_terraform_hetzner_cloud")

  network_region = "eu-central"

  control_plane_nodepools = [
    {
      name        = "control-plane",
      server_type = "cax11",
      location    = "fsn1",
      labels      = [],
      taints      = [],
      count       = 3
    }
  ]

  agent_nodepools = [
    {
      name        = "agent-1",
      server_type = "cx21",
      location    = "fsn1",
      labels = [
        "run=application",
      ],
      taints = [],
      count  = 1,
    },
    {
      name        = "agent-2",
      server_type = "cx21",
      location    = "fsn1",
      labels = [
        "run=packages",
      ],
      taints = [],
      count  = 1,
    },
    {
      name        = "agent-3",
      server_type = "cx21",
      location    = "fsn1",
      labels = [
        "node.kubernetes.io/server-usage=storage",
        "node.longhorn.io/create-default-disk=true"
      ],
      taints               = [],
      count                = 1,
      longhorn_volume_size = 0
    }
  ]

  load_balancer_type     = "lb11"
  load_balancer_location = "fsn1"

  dns_servers = [
    "1.1.1.1",
    "8.8.8.8",
    "2606:4700:4700::1111",
  ]

  microos_x86_snapshot_id = var.microos_x86_snapshot_id
  microos_arm_snapshot_id = var.microos_arm_snapshot_id

  create_kubeconfig = true
  export_values     = true

  extra_firewall_rules = [
    # {
    #   description     = "For Postgres"
    #   direction       = "in"
    #   protocol        = "tcp"
    #   port            = "5432"
    #   source_ips      = ["0.0.0.0/0", "::/0"]
    #   destination_ips = [] # Won't be used for this rule
    # },
    {
      description     = "To Allow ArgoCD access to resources via SSH"
      direction       = "out"
      protocol        = "tcp"
      port            = "22"
      source_ips      = [] # Won't be used for this rule
      destination_ips = ["0.0.0.0/0", "::/0"]
    }
  ]

  enable_longhorn        = true
  longhorn_replica_count = 2

  longhorn_values = <<EOT
defaultSettings:
  createDefaultDiskLabeledNodes: true
  defaultDataPath: /var/longhorn
  node-down-pod-deletion-policy: delete-both-statefulset-and-deployment-pod
persistence:
  defaultFsType: ext4
  defaultClassReplicaCount: 1
  defaultClass: true
  reclaimPolicy: Retain
  EOT
}

output "kubeconfig" {
  value     = module.kube-hetzner.kubeconfig
  sensitive = true
}

Run Setup

terraform init
terraform apply

Finish!

Upon completion of the Terraform process, a file named k3s_kubeconfig.yaml will be generated, containing the configuration for your K3S cluster.

Connect With kubectl

Verify your setup by connecting with kubectl using the generated kubeconfig file:

kubectl get namespaces --kubeconfig=k3s_kubeconfig.yaml
kubectl get nodes --kubeconfig=k3s_kubeconfig.yaml

Resources that will be created on the Hetzner cloud

The Terraform setup will create various resources on Hetzner Cloud, including Snapshots, Firewall, SSH Key, Network, Load Balancer, and Servers.

GitHub Template

Template code in my GitHub repository https://github.com/MahdadGhasemian/kubernetes-hetzner-lac-template

Cost Estimation for this setup

update (Mar 25 2024)