Cloud Hosting a Simple Git Server with Kubernetes

Let’s host a Git server with a cloud-based Kubernetes engine. The server will support git-shell over an authenticated SSH connection.

Please note that there is no CI/CD tooling or project management framework packaged with a Git server — just in case you landed here looking for something like GitLab.

I should also address that if all you want to do is host a Git server, then a Kubernetes cluster is overkill. I am using a Kubernetes approach since I already have a cluster that orchestrates other personal projects.

Pros & Cons

I decided to host a Git server in the cloud for two reasons. Firstly, the majority of my home projects did not benefit from the vast project and user features of online version control platforms. Secondly, I had become wary of the future of my data on freemium code hosting platforms.

I have perused the Git server market: I have managed GitLab servers, used GitHub for a few years, had a brief developer experience using Azure Repos, and tried out projects like Gitea. However, even the wonderfully simplistic Gitea project is just too feature-rich for me, so let’s go through the pros and cons of the approach I will take in this article.

Pros

1. Minimize web attack surface. By only providing an SSH interface, HTTP based intrusion opportunities are minimized i.e. A bad actor will not gain access to source code by stealing a developer’s web browser session tokens.
2. Simple backups. Git repositories exist in a persistent volume claim. Schedule a backup operation for the volume’s contents and store responsibly.
3. Separate valuable project intelligence from source code. There are benefits to using version control solutions that bring project management tools like support tickets and CI/CD pipeline history right next to source code, but if you are here then that is likely not what you want for security or business reasons. A simple Git server will isolate code from secrets management, developer details, and whatever AI abomination that might like to comb through your intellectual property — I hope a talking toaster does not make me pay for my ways someday.

Cons

1. Something else to maintain and monitor. As a Git server administrator, be sure to watch your logs for suspicious connection attempts and be wary of who has the keys to your server.
2. No user roles. Any developer given access to the server via SSH key could execute arbitrary Git operations if you use an authorized keys file without Pluggable Authentication Modules (PAM).
3. No GUI for standard Git operations. There are plenty of visual client-side tools that can be downloaded for free for tasks like merging and branch visualization. The real draw here is that there is no “Create Repository” button. To create a repository, you will need to get a shell into your Git server and create an empty Git repository using shell commands. I will provide an example later in the article.

Prerequisites

You will need a working knowledge of Git, Docker, and Kubernetes with access to a container registry and a Kubernetes cluster on the web.

Configure & Host

We need to accomplish the following:

1. Store Git repositories
2. Store authorized SSH keys for Git users
3. Deploy a git-shell service for authenticated SSH connections
4. Expose the service to the web (you will be on your own here, since this step varies depending on your cloud architecture choices)

Storage

A persistent volume claim for repositories

I do not have many repositories, and they are lightweight. I will use my cloud service's minimum claim size of 1Gi along with its classic storage class offering.

The name is derived from its intended location in the deployed pod: /home/git/repositories.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: simple-git-server-git-repositories
  labels:
    app: simple-git-server
spec:
  storageClassName: csi-cinder-classic
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

A persistent volume claim for user credentials

The Git server is an OpenSSH server using git-shell, so the simplest authentication solution is to just register our users by their public SSH keys in the ./ssh/authorized_keys file.

Consider using a Kubernetes secret to hold the contents of the authorized keys file if you do not intend on interacting with these credentials as a file. I have a particular use case where an automated process manipulates this file as I rotate my keys, so I use a persistent volume claim to contain it.

The name is derived from its intended location in the deployed pod: /home/git/.ssh.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: simple-git-server-git-ssh
  labels:
    app: simple-git-server
spec:
  storageClassName: csi-cinder-classic
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

Docker image and SSH daemon configuration

The version control service is based on the Alpine Docker image. Worried about CVE-2019–5021? Remember to always use a secure version of Alpine.

Below is a minimal Dockerfile and its corresponding SSH daemon configuration:

FROM alpine:3.17

RUN apk add git openssh shadow

# That's a bad password! However, password login will be disabled in the sshd_config step.
RUN adduser git -s /usr/bin/git-shell; \
    echo 'git:git' | chpasswd

# Generate root ssh keys, these are required as host keys even if root login is disabled
RUN ssh-keygen -A

# Git tastes: I remove the message of the day and set my default branch to main
RUN rm /etc/motd
RUN git config --global init.defaultBranch main

EXPOSE 22
COPY sshd_config /etc/ssh/sshd_config
CMD /usr/sbin/sshd -D -e

In the same directory as the Dockerfile above, create a file called sshd_config with the following settings.

# Critical settings
PermitRootLogin                 no  # NO root login
PasswordAuthentication          no  # NO password login
ChallengeResponseAuthentication no  # NO asking for password login after a failed public key login

# YES to public key authentication using your authorized keys file
PubkeyAuthentication            yes 
AuthorizedKeysFile .ssh/authorized_keys

# I disabled what I know I will not use and went with the internal sftp option
AllowTcpForwarding no
GatewayPorts       no
X11Forwarding      no
Subsystem sftp internal-sftp

Build and push this Docker image as simple-git-server to your container registry for the next step.

Kubernetes service and deployment

Tie the Docker image and storage volumes together into a deployed service.

Remember to choose names, namespaces, and labels wisely, these files are minimal examples.

apiVersion: v1
kind: Service
metadata:
  name: simple-git-server
  labels:
    app: simple-git-server
spec:
  ports:
    - port: 22
  selector:
    app: simple-git-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: simple-git-server
spec:
  strategy:
    type: Recreate
  replicas: 1
  selector:
    matchLabels:
      app: simple-git-server
  template:
    metadata:
      labels:
        app: simple-git-server
    spec:
      initContainers:
        - name: take-git-repositories-dir-ownership
          image: <YOUR CONTAINER REGISTRY PREFIX>/simple-git-server:latest
          command:
          - chown
          - -R
          - git:git
          - /home/git/repositories
          volumeMounts:
            - mountPath: "/home/git/repositories"
              name: simple-git-server-git-repositories
              readOnly: false
      containers:
        - image: <YOUR CONTAINER REGISTRY PREFIX>/simple-git-server:latest
          name: version-control-service
          ports:
            - containerPort: 22
          volumeMounts:
            - mountPath: "/home/git/.ssh"
              name: simple-git-server-git-ssh
              readOnly: false
            - mountPath: "/home/git/repositories"
              name: simple-git-server-git-repositories
              readOnly: false
      imagePullSecrets:
        - name: <YOUR CONTAINER REGISTRY CREDENTIALS SECRET NAME>
      volumes:
        - name: simple-git-server-git-repositories
          persistentVolumeClaim:
            claimName: simple-git-server-git-repositories
        - name: simple-git-server-git-ssh
          persistentVolumeClaim:
            claimName: simple-git-server-git-ssh

There is an init containers entry that grants ownership of the repositories directory to the git user, this allows git-shell to operate as the user git on those repositories on behalf of the SSH-connected user.

Expose this service to the web

This step will vary depending on how your cluster is organized. The takeaway is that port 22 of simple-git-server must be reachable on the web.

Using the Git Server

Once the Git server is running, let’s create a repository on the server and clone it on the client. Get a root shell into the running pod via kubectl — and remember to use sh since Alpine Linux does not come with bash.

Switch to the git user using git-shell since you will need to create directories. Change the working directory to the mounted location of git-server-git-repositories and create a bare repository.

su -s /bin/sh git

cd /home/git/repositories
mkdir hello-world.git
git init --bare hello-world.git

The Moment of Truth

Have you populated your /home/git/.ssh/authorized_keys file yet?

If not, you may access the simple-git-server-git-ssh volume claim mounted at /home/git/.ssh in the same manner as you did for the simple-git-server-git-repositories volume claim above. You might quickly copy your development machine’s public SSH key into /home/git/.ssh/authorized_keys .

With authorized keys squared away, try cloning hello-world from the development machine.

git clone git@<GIT SERVER BASE URL>:repositories/hello-world.git

I would be surprised if everything worked on your first try, but if it did then congratulations. You are now the administrator of a cloud-hosted git server using Kubernetes.