Build time-related tasks with Kubernetes

Deploy time-bound microservice with Cronjob on Google Kubernetes Engine

HumansOfCarbon

Imagine these two superhumans of Carbon being subjected to moving a task such as the size of this mountain and having to do so every 12 hours daily. In this article, I demonstrated how we use Kubernetes on Google Cloud Platform to deliver time-based tasks while saving cost and compute resources.

Case Scenario

A common time-based infrastructure engineering task at Carbon is to implement a solution that periodically exports data from a MySQL data source to a Warehouse on Google BigQuery for possible data analytics service.

Cost impact

Spinning up a dedicated instance to get this done would cost about $194.18/month for one n1-standard-8 instance type. If you have an existing GKE cluster with running nodes, you can leverage on those nodes and spin a time-bound pod to get the same job done with less than $10/month expense.

About Kubernetes?

Kubernetes allows you to automate the deployment, scaling, and management of containerized applications. Containerizing an application requires a base image that can be used to create an instance of a container.

Kubernetes CronJob

CronJob is a Kubernetes object that creates jobs in a repeatable manner to a defined schedule. CronJobs perform finite, time-related tasks that run once or repeatedly at a time that you specify using Job objects to complete their tasks.

The Demo

1. Prerequisite: A GCP account. It is free. 😎
2. Create a Project within your GCP account.

Getting Started

• Activate Google Cloud Shell. In the GCP console, on the top right toolbar, click the Open Cloud Shell button.
• Click Continue. Wait for the environment to provision.

Create a Google Kubernetes Engine cluster

#set the environment variable for the zone and cluster name
export my_zone=us-central1-a 
export my_cluster_name=demo-cluster
#type the following command to create a Kubernetes cluster
gcloud container clusters create $my_cluster_name --num-nodes 2  --enable-ip-alias --zone $my_zone
#configure access to your cluster for the kubectl command-line tool, using the following command
gcloud container clusters get-credentials $my_cluster_name --zone $my_zone

creating GKE cluster

Define and deploy a CronJob manifest

cronjob manifest: sample-cronjob.yaml

Execute the following command to deploy the CronJob manifest:

kubectl apply -f sample-cronjob.yaml

create the cronjob

Introspection:

# To get a list of the Jobs in the cluster, 
kubectl get jobs
# To check the status of this Job, execute the following command, where [cronjob_name] is the name of your job:
kubectl describe job [cronjob_name]
# View the output of the Job by querying the logs for the Pod. Replace [POD-NAME] with the name of the Pod you recorded in the last step.
kubectl logs [POD-NAME]

The Output of Job executed by each pod can be seen in the images below;

cronjob executions

Clean up

# To delete all these jobs, execute the following command:
kubectl delete cronjob hello

Conclusion

In traditional deployments on servers, engineers make use of cronjobs on Linux or scheduled tasks on Windows to execute time-relevant tasks. CronJob on GKE also offers equal benefits for organizations orchestrating microservices with Kubernetes while packaging jobs in Docker containers.

Enjoy!!!