How to deploy gRPC service to AWS EKS

gRPC meets AWS EKS

gRPC takes a major role in the high-performance communication protocol. It’s the top layer of HTTP/2 also with QUIC or HTTP/3 in the future. We take advantage of Bidirectional streaming, e.g., transferring images between client and ML server in a single connection.

Deploying the gRPC service for K8s to AWS EKS is still new for the community. So, we would like to share the process to deploy gPRC to the EKS cluster.

Prerequisites

The following bits of knowledge are required before deployment:

• EKS cluster with eksctl & kubectl
• Helm V3
• AWS IAM Permissions
• Route 53 Domain “test-grpc.com” in this case
• AWS Certificate Manager

Get Started

Create EKS cluster

Create EKS cluster with your name & region. This takes approximately 15 minutes.

export AWS_CLUSTER_NAME=grpc-cluster
export AWS_REGION=us-west-2
export K8S_VERSION=1.21
eksctl create cluster \
  --name=${AWS_CLUSTER_NAME} \
  --version=${K8S_VERSION} \
  --managed --nodes=1 \
  --region=${AWS_REGION} \
  --node-type t3.small \
  --node-labels="lifecycle=OnDemand"

Setup AWS Load Balancer Controller (ALB)

Due to the installation not yet settled, please follow this guideline for the latest update from AWS team to install ALB on EKS.

Deploy sample gRPC server manifests

Deploy all the manifests from GitHub.

kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/examples/grpc/grpcserver-namespace.yaml 
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/examples/grpc/grpcserver-service.yaml
 
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/examples/grpc/grpcserver-deployment.yaml

Confirm that all resources were created in READY and STATUS column.

kubectl get -n grpcserver all

Custom ingress service

Download the ingress manifest.

wget https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/examples/grpc/grpcserver-ingress.yaml

An important step to custom ingress manifest we download before

1. spec > rules > host — Change host from grpcserver.example.com to test-grpc.com in this case.
2. metadata > annotations — Add your cluster public subnets to alb.ingress.kubernetes.io/subnets key for example:

metadata:
  annotations:
    alb.ingress.kubernetes.io/subnets: subnet-xx,subnet-xx,subnet-xx
    alb.ingress.kubernetes.io/actions.ssl-redirect: …
…

Deploy ingress we custom before

kubectl apply -f grpcserver-ingress.yaml

Wait a few minutes for provisioning and check the ALB address with

kubectl get ingress -n grpcserver grpcserver

# sample output
NAME       CLASS  HOSTS  ADDRESS                     
grpcserver <none> *      k8s-grpcserv-xx.us-west-2.elb.amazonaws.com
PORTS   AGE
80      2m32s

Add A record in Route 53

Important step:

1. Copy ALB address (k8s-grpcserv-xxxx) from previous step
2. Add ALB address to A record for “test-grpc.com” in Route 53
3. Do not choose “dualstack.k8s-grpcserv-xxxx” just put plain text from the previous output for example

Add ALB address to A record

Finally, test this sample gRPC server (you may wait for DNS provisioning)

docker run --rm --it --env BACKEND=test-grpc.com placeexchange/grpc-demo:latest python greeter_client.py
# sample response
Greeter client received: Hello, you!

The End

If there’re any problems feel free to leave a comment below. You may apply this concept to the Spot node group that autoscale from zero to desire your workload to optimize your cost as we describe in this post

How we reduce 60% cost for ML cluster with K8s

About us

We’re a tech startup based in Southeast Asia. We create an AR Cloud Platform using our VPS technologies. Make The Metaverse happen in the real world.

Graffity Technologies