How to create a full-featured Kubernetes platform for running micro-services
I have been writing a series of articles that explain how to create a full-featured Kubernetes platform. The aim is to create a cluster with all the functions you need to operate and maintain your microservices. In this article I bring it all together by referencing the articles in the order they should be followed to create the platform.
Introduction
This series of articles brings together a number of technologies to create a Kubernetes cluster and associated non-cluster servers that creates a platform for running one or more microservices.
In all my solutions, I use an Australian cloud provider, Binary Lane, which provides basic hosting services. Not only are they cost effective, they are reliable. By creating solutions based on their no-frills service, it means you learn a lot whilst creating solutions with little to no lock in. This is unlike other suppliers such as AWS, Azure and Google Cloud, which make life easy by providing sophisticated and complete services that lead to provider lock in.
Note that all my virtual private servers are based on Ubuntu 22.04 and I cannot guarantee that the solutions wil work on other operating systems.
Technologies
The technologies used in the overall solution include:
- APISIX — a free open-source API Gateway controlling access into the cluster
- Grafana — a log and metric visualisation platform
- Helm — a package manager for Kubernetes
- Keycloak — an Identity and Access Management solution
- Kong — an API gateway controlling access into the cluster
- Kubernetes — a container orchestration platform
- Let’s Encrypt — a CA that provides PKI certificates for free
- Loki — a log aggregation system for Grafana
- MinIO — an S3 compatible object storage
- NGINX — a reverse proxy load balancer
- NFS — a Network File System for providing persistency
- OpenVPN — providing secure access into your Virtual Private Cloud (VPC)
- PostgreSQL — a database to provide persistency
- Prometheus — providing visibility of metrics from within the cluster and servers external to the cluster
- Promtail — collects logs and sends them to Loki
- Slack — a channel for receiving alerts
Solution description and references
The architecture shown in the diagram above is built up over a series of articles that are designed to be followed in order. This series is shown in the list below (Note: those without a link are in the process of being created):
- My guide to Public Key Infrastructure (PKI) — optional
- Introduction to creating a CA on Debian 11
and
Creating a CA with easyrsa on Debian 12 — optional - Guide to using OpenVPN to access servers in your VPC
- Automatic creation of Kubernetes cluster on Binary Lane
- Adding persistent storage to your Kubernetes cluster
- Adding observability to a Kubernetes cluster using Prometheus
- Adding Grafana visualisation to a Kubernetes cluster with Prometheus
- Object Storage in your Kubernetes cluster using MinIO
- Improve observability by adding logs to Grafana on a Kubernetes cluster
- Simple service for testing Kubernetes configurations
- Why do I need an API Gateway on a Kubernetes cluster
and
Installing APISIX API Gateway on your Kubernetes Cluster
and
Integrating Keycloak IAM with APISIX to secure your Kubernetes Services
and
Using Kong to access Kubernetes services, using a Gateway resource with no cloud provided LoadBalancer - Adding a Postgres High-Availability database to your Kubernetes cluster
- Backing up your CloudNativePG cluster to Object Storage using Minio
- Adding a wildcard Let’s Encrypt certificate to your server without a web server
and
Using a wildcard certificate within your Kubernetes cluster - Adding TLS connectivity to MinIO
- Introduction to Vault to provide secret management in your Kubernetes cluster
and
Enabling TLS on your Vault cluster on Kubernetes - What are the 3 different layers of security that control user authorisation and why we have them
and
What is OIDC and OAuth 2.0 and why are they used
and
How to install Keycloak IAM with APISIX on your Kubernetes cluster
Spring Boot Solution
As well as the solution mentioned above, there is another sequence of articles that looks at creating a solution for running a Spring Boot application on Kubernetes with a full Continuous Integration / Continuous Deployment (CI/CD) pipeline. It also includes automatic creation of the infrastructure.
This sequence of articles is called:
Spring Boot CI/CD on Kubernetes using Terraform, Ansible and GitHub.
Parts of this series can be used in conjunction with the solution above to provide even greater functionality. It includes:
- Part 1: Introduction to creating a CI/CD pipeline on Kubernetes in the cloud
- Part 2: Setting up the project for automated provisioning of Binary Lane infrastructure
- Part 3: Automatic creation of the cloud infrastructure using Terraform
- Part 4: Configuring servers using Ansible
- Part 5: Creating a Kubernetes cluster
- Part 6: Creating a Persistent Volume and connecting it to a postgreSQL database
- Part 7: Creating a Spring Boot Application to add to your cluster
- Part 8: Adding a Spring Boot Application to your cluster
- Part 9: Accessing a Spring Boot application using a Kubernetes Service
- Part 10: Use Terraform, Ansible and GitHub Actions to automate running your Spring Boot application on Kubernetes
- Part 11: Deploying Spring Boot Application with Continuous deployment
- Part 12: Using ArgoCD to deploy your Spring Boot Application
In these articles, the code can be found on my github account.
Summary
This article has presented two sets of articles that build platforms for your microservices.
The first is a fully featured Kubernetes platform that can run any microservice application.
The second is specifically designed to demonstrate how to implement Spring Boot on Kubernetes with a fully automatic CI/CD pipeline.
Note that neither of these solutions is production ready. You need to assess your availability and reliability targets as well as your security requirements before basing any solution on these articles.
