Build time-related tasks with Kubernetes

Deploy time-bound microservice with Cronjob on Google Kubernetes Engine

Olawale Olaleye
Dec 26, 2019 · 3 min read
HumansOfCarbon

Imagine these two superhumans of Carbon being subjected to moving a task such as the size of this mountain and having to do so every 12 hours daily. In this article, I demonstrated how we use Kubernetes on Google Cloud Platform to deliver time-based tasks while saving cost and compute resources.

Case Scenario

A common time-based infrastructure engineering task at Carbon is to implement a solution that periodically exports data from a MySQL data source to a Warehouse on Google BigQuery for possible data analytics service.

Cost impact

Spinning up a dedicated instance to get this done would cost about $194.18/month for one n1-standard-8 instance type. If you have an existing GKE cluster with running nodes, you can leverage on those nodes and spin a time-bound pod to get the same job done with less than $10/month expense.

About Kubernetes?

Kubernetes allows you to automate the deployment, scaling, and management of containerized applications. Containerizing an application requires a base image that can be used to create an instance of a container.

Kubernetes CronJob

CronJob is a Kubernetes object that creates jobs in a repeatable manner to a defined schedule. CronJobs perform finite, time-related tasks that run once or repeatedly at a time that you specify using Job objects to complete their tasks.


The Demo

  1. Prerequisite: A GCP account. It is free. 😎
  2. Create a Project within your GCP account.

Getting Started

  • Activate Google Cloud Shell. In the GCP console, on the top right toolbar, click the Open Cloud Shell button.
  • Click Continue. Wait for the environment to provision.

Create a Google Kubernetes Engine cluster

#set the environment variable for the zone and cluster name
export my_zone=us-central1-a
export my_cluster_name=demo-cluster
#type the following command to create a Kubernetes cluster
gcloud container clusters create $my_cluster_name --num-nodes 2 --enable-ip-alias --zone $my_zone
#configure access to your cluster for the kubectl command-line tool, using the following commandgcloud container clusters get-credentials $my_cluster_name --zone $my_zone
creating GKE cluster

Define and deploy a CronJob manifest

cronjob manifest: sample-cronjob.yaml

Execute the following command to deploy the CronJob manifest:

kubectl apply -f sample-cronjob.yaml
create the cronjob

Introspection:

# To get a list of the Jobs in the cluster, 
kubectl get jobs
# To check the status of this Job, execute the following command, where [cronjob_name] is the name of your job:kubectl describe job [cronjob_name]# View the output of the Job by querying the logs for the Pod. Replace [POD-NAME] with the name of the Pod you recorded in the last step.kubectl logs [POD-NAME]

The Output of Job executed by each pod can be seen in the images below;

cronjob executions

Clean up

# To delete all these jobs, execute the following command:kubectl delete cronjob hello

Conclusion

In traditional deployments on servers, engineers make use of cronjobs on Linux or scheduled tasks on Windows to execute time-relevant tasks. CronJob on GKE also offers equal benefits for organizations orchestrating microservices with Kubernetes while packaging jobs in Docker containers.

Enjoy!!!

Carbon Engineering & Data Science

All things Engineering and Data Science at Carbon.

Olawale Olaleye

Written by

MultiCloud Specialist | SRE | DevOps | IT Infrastructure & Security Specialist.

Carbon Engineering & Data Science

All things Engineering and Data Science at Carbon.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade