Stories by Naveen on Medium

Amazon EKS: Deploying a Go Server with AWS Load Balancer Controller

Naveen — Thu, 02 Nov 2023 14:06:21 GMT

In my previous post , we created a basic Go application, generated a Docker image, and deployed it to our local minikube k8s cluster. In this post, let’s take that same application and deploy it on Amazon’s EKS.

Prerequisites

Familiarity with the previous post is recommended but not mandatory.
Install the AWS CLI and configure your account, either through a named profile or default. AWS CLI Installation Guide.
Install eksctl, a tool that simplifies EKS cluster management. eksctl Installation.
Set up kubectl to access your EKS cluster. Install kubectl.

The Why and What of EKS?

Working with Kubernetes involves great abstraction and greater complexity. Managing a Kubernetes cluster involves tasks like upgrades, handling certificates, managing etcd and master node crashes. EKS, as a managed platform, abstracts the control plane components and simplifies cluster management, allowing you to focus on worker nodes. See a comparison of Self-Managed vs. Managed Platforms here.

EKS Cluster

We will create an EKS cluster, and our containers will run on AWS Fargate, a managed compute engine. Fargate eliminates the need to manage infrastructure, handling tasks like scaling and patching.

Create an EKS Cluster

eksctl create cluster --name go-server-cluster --region  --fargate --profile

This command creates a cluster in the specified region. The — fargate option creates a default Fargate profile. The command also sets up a new VPC, and brings up the entire cluster.

aws eks update-kubeconfig --name go-server-cluster --region  --profile

Configures kubectl on our local machine to interact with the EKS cluster by creating and switching to a new kubectl context.

eksctl create fargateprofile --cluster go-server-cluster --region  --profile  --name go-server-alb --namespace goserver-2023

Creates a new Fargate profile for deploying pods onto Fargate.

# go-server.yaml

---
apiVersion: v1
kind: Namespace
metadata:
  name: goserver-2023
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: goserver-2023
  name: goserver-deployment
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: goserver-2023
  replicas: 5
  template:
    metadata:
      labels:
        app.kubernetes.io/name: goserver-2023
    spec:
      containers:
      - image: knav33n/go-server:latest
        imagePullPolicy: Always
        name: goserver-2023
        ports:
        - containerPort: 3333
---
apiVersion: v1
kind: Service
metadata:
  namespace: goserver-2023
  name: goserver-service
spec:
  ports:
    - port: 80
      targetPort: 3333
      protocol: TCP
  type: NodePort
  selector:
    app.kubernetes.io/name: goserver-2023
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: goserver-2023
  name: goserver-ingress
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: goserver-service
              port:
                number: 80

kubectl apply -f go-server.yaml

Creates various Kubernetes resources as defined in the ‘go-server.yaml’ file. The public image `knav33n/go-server:latest` is pulled from Dockerhub.

Configuring the Policy and Service Account for the ALB

Amazon EKS provides support for IAM Roles for Service Accounts, allowing cluster operators to associate AWS IAM Roles with Kubernetes Service Accounts. This facilitates fine-grained permission management for applications running on EKS that utilize various AWS services. These services may include components like the AWS Load Balancer controller or ExternalDNS.

eksctl utils associate-iam-oidc-provider --cluster $cluster_name --approve --profile

Enables the IAM OIDC Provider, which is not enabled by default.

curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/install/iam_policy.json

Fetches the necessary policy document.

aws iam create-policy --policy-name AWSLoadBalancerControllerIAMPolicy --policy-document file://iam_policy.json --profile

Creates a policy in AWS with the specified name and the policy document we fetched.

eksctl create iamserviceaccount --cluster=go-server-cluster --namespace=kube-system --name=aws-load-balancer-controller --profile  --role-name AmazonEKSLoadBalancerControllerRole --attach-policy-arn=arn:aws:iam:::policy/AWSLoadBalancerControllerIAMPolicy --approve

Creates an IAM Service Account for the AWS Load Balancer controller.

The AWS Load Balancer controller (LBC) manages AWS Network Load Balancer (NLB) and Application Load Balancer (ALB) resources. It continuously monitors Kubernetes service and ingress resources, configuring AWS resources accordingly.

Add Controller to Cluster

To install and configure the AWS Load Balancer controller as the Ingress controller, use the following commands.

helm repo add eks https://aws.github.io/eks-charts
helm repo update eks
helm install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system --set clusterName=go-server-cluster --set serviceAccount.create=false --set serviceAccount.name=aws-load-balancer-controller --set region= --set vpcId=

Upon completion, the service will be assigned an external IP address, and our Go server will be accessible via the internet.

kubectl get svc -n goserver-2023

To find the external IP address of the service.

With the ingress mapped to forward requests from port 80 to 3333, we can access our server using curl , which should yield a 200 response.

Clean Up

Use the following commands to clean up the AWS resources.

eksctl delete cluster --name go-server-cluster --profile

Delete the EKS cluster

eksctl delete iamserviceaccount -c go-server-cluster --name aws-load-balancer-controller --profile

Delete the IAM service account

aws iam detach-role-policy --role-name AmazonEKSLoadBalancerControllerRole --policy-arn arn:aws:iam:::policy/AWSLoadBalancerControllerIAMPolicy --profile

Detach the role policy

aws iam delete-role --role-name AmazonEKSLoadBalancerControllerRole --profile

Delete the role

aws iam delete-policy --policy-arn arn:aws:iam:::policy/AWSLoadBalancerControllerIAMPolicy --profile

Delete the policy

Helpful Links

Deploying a Basic Go Server Using Helm Locally

Naveen — Wed, 18 Oct 2023 10:01:43 GMT

Deploying a Go Server Using Helm Locally

What We Are Going to Do

We will deploy a Go application with 2 endpoints onto a Kubernetes (k8s) cluster using Helm. The process involves creating a Docker container for the application, pushing the image to Dockerhub, and configuring our Helm chart to fetch the Go application from Dockerhub.

Prerequisites

To follow this tutorial effectively, you should have some knowledge of Kubernetes. Although Helm abstracts away a lot of YAML configuration, familiarity with Kubernetes will help you appreciate the simplification. Here are the prerequisites:

Go: Install Go by following the instructions at Go Installation.
kubectl: Install kubectl by visiting Install kubectl.
Microk8s: You can set up a Kubernetes cluster on your local machine using microk8s. Refer to Microk8s Documentation for installation and Getting Started for initial setup.

// if you plan on using the official kubectl, install kubectl first and then install Microk8s

sudo snap install microk8s
sudo usermod -a -G microk8s $USER

cd $HOME
mkdir .kube
cd .kube
microk8s config > config

sudo chown -f -R $USER ~/.kube

Helm: Install Helm using a script or package manager relevant to your operating system. Refer to Helm Installation.

What Is Helm? Why Helm?

Helm is a Kubernetes package manager designed to simplify the lifecycle management of Kubernetes applications. Think of it as a package manager for Kubernetes, similar to apt, homebrew, or npm. Instead of binary apps or JS modules, Helm deals with entire applications that can be deployed on a Kubernetes cluster. You can search for packages using `helm search hub ` after installing Helm. Additionally, Helm allows you to add repositories to download private Kubernetes packages.

If you’ve encountered the complexity of Kubernetes, you know that all resources are typically declared in YAML. Helm simplifies this by generating the necessary YAML with a single command. This makes deploying and managing applications more straightforward. It’s especially useful for maintaining multiple environments and facilitating updates and rollbacks.

The Go Server

Let’s build a simple Go app. Create a directory and a file named `main.go`.

// main.go

package main

import (
 "errors"
 "fmt"
 "io"
 "net/http"
 "os"
)

func getRoot(w http.ResponseWriter, r *http.Request) {
 fmt.Printf("got / request\n")
 io.WriteString(w, "https://naveenkumar.dev 🔥🎇🎆🧨\n")
}

func getHello(w http.ResponseWriter, r *http.Request) {
 fmt.Printf("got /hello request\n")
 io.WriteString(w, "Hello from Helm!\n")
}

func main() {
 http.HandleFunc("/", getRoot)
 http.HandleFunc("/hello", getHello)

 err := http.ListenAndServe(":3333", nil)
 if errors.Is(err, http.ErrServerClosed) {
  fmt.Printf("server closed\n")
 } else if err != nil {
  fmt.Printf("error starting server: %s\n", err)
  os.Exit(1)
 }
}

This app provides two endpoints: `/` and `/hello`, exposed on port `3333`. Initialize the Go module with `go mod init go-server`, and run the application with `go run main.go`. Use `curl localhost:3333/hello` to verify that it responds with “Hello from Helm!”.

The Go Server Container

Let’s create a Dockerfile for the Go application to build a Docker image and push it to Dockerhub.

# Dockerfile

FROM golang:1.21.3-alpine3.18
WORKDIR /app
COPY go.mod ./
RUN go mod download
COPY *.go ./
RUN CGO_ENABLED=0 GOOS=linux go build -o /go-server
EXPOSE 3333
CMD ["/go-server"]

Build the image and run it locally using the following commands:

docker build --tag go-server .
docker run -d -p 3333:3333 go-server

Use `curl localhost:3333/hello` to verify that it still responds with “Hello from Helm!”. Next, tag the image and push it to Dockerhub since Helm and Kubernetes fetch images from Dockerhub. Log in to Dockerhub via the CLI before pushing the image:

docker login
docker tag go-server /go-server:latest
docker push /go-server

Helm Chart

Navigate to a workspace directory and execute the following command:

helm create go-server

The `helm create` command generates a template needed for any single-tier application. Open the `values.yaml` file generated by the previous command and update the `repository` field under `image` to `/go-server:latest`. Additionally, change the `type` of service to `NodePort` and the port to 3333 (matching the application’s port).

You’re almost done! Deploy your application to the Microk8s cluster with the following command. Ensure you’ve stopped the previously started Docker container using `docker stop `.

helm install go-server-helm go-server

You can verify what’s created in the Kubernetes cluster with these commands:

helm list -a
kubectl get po -w
kubectl get svc
curl 
curl :3333
curl :3333/hello

`curl localhost:3333/hello` should respond with “Hello from Helm!” once more.

Cleanup

Use `helm uninstall go-server-helm` to remove the application from the cluster. To remove the Docker container and image:

docker rm  // for the container
docker rmi  // for the image

Helpful Links

My Journey through — The Cloud Resume Challenge

Naveen — Wed, 28 Jun 2023 08:02:07 GMT

My Journey through — The Cloud Resume Challenge

Cloud Resume Challenge

Links -

Site: https://naveenkumar.dev
Repo: https://github.com/knav33n/naveenkumar.dev

As someone who loves to learn by doing, I was actively exploring ways to gain practical experience in cloud-related skills. That’s when I stumbled upon The Cloud Resume Challenge. Even though I have prior experience in software development, I was genuinely amazed at how much I learned through this challenge. In a nutshell, the challenge invites you to deploy an application on a cloud provider of your choice. The flexibility to choose any provider intrigued me, and given my familiarity with AWS, I decided to go with it.

Creating the static site was a breeze for me, as I’ve been working with front-end development since I first started in software. Instead of opting for a comprehensive resume, I decided to go with a simpler introductory/portfolio page. Deploying the static site proved to be relatively straightforward using the user interface. First, I needed to generate a certificate for my domain through ACM (AWS Certificate Manager), which was then validated through a CNAME entry. Next, I set up an S3 bucket and created a CloudFront distribution that linked to the S3 bucket, making sure to include the alternate domain I had purchased. I then updated the bucket policy with the one provided by CloudFront. At this point, I could access the site using the domain generated by CloudFront. To further connect everything together, I created a CNAME entry in Route53 to point my domain to the CloudFront domain. Voila! The site was now fully accessible through my own domain name.

The next step involved writing a lambda function to track the number of views and store them in DynamoDB. This part took some time since I was relatively new to working with Python and it was my first experience with Lambda and DynamoDB. To ensure the lambda function could access the DynamoDB table, I created a relevant role that granted the necessary read and write permissions.

Lambda provided a function URL that could be utilized to call it. Curious to explore further, I decided to experiment by using a subdomain of mine to invoke this lambda function. This required setting up an API Gateway and configuring it to forward requests to the lambda function(though this is not in the final repo). To ensure security, I updated the CORS settings of the lambda function, ensuring that it could only be called from my own domain.

Now, whenever the page loads, an AJAX call from the front-end triggered the update of the DynamoDB table. The response from this update contained the updated view count, which was then displayed on the UI. It was truly rewarding to witness the views being dynamically updated in real time.

The next crucial step was to incorporate monitoring capabilities. To achieve this, I proceeded to set up CloudWatch alarms to keep a close eye on any errors or invocations of the lambda function. In order to receive timely notifications, I configured SNS topics and created an additional Lambda function with the necessary role. This Lambda function was specifically designed to register and publish these alarms to a designated channel within a Slack workspace. By utilizing webhooks, I established a seamless integration between CloudWatch and Slack, enabling real-time updates and notifications.

For source control, I relied on GitHub, although I must admit that GitHub Actions was a new concept for me. I encountered numerous failed workflows along the way, which was a bit frustrating. To make matters more confusing, there were instances when GitHub Actions experienced downtime, adding to the challenges I faced.

Next, I delved into using Terraform to provision the necessary infrastructure. It was a whole new learning experience, and I dedicated hours upon hours to understanding how it functions. I scoured through documentation, learning the ins and outs of deploying the desired resources. It involved a lot of trial and error, and there were countless moments of searching for solutions.

But eventually, after much persistence and effort, I managed to provision everything I needed using Terraform. From setting up S3 buckets to configuring CloudWatch alarms, I successfully achieved the desired infrastructure setup. It was truly fulfilling to see everything fall into place after overcoming the initial learning curve and persisting through the challenges.

In conclusion, this journey has been nothing short of incredible, filled with invaluable learning experiences. I am immensely grateful to Forrest Brazeal for crafting such an amazing challenge. It has pushed me to explore new territories, expand my skill set, and discover the boundless possibilities of cloud-related technologies.

Helpful links -