If you’ve been in the IT world in the past 20 years, you’ve probably had to deal with some sort of job scheduler. The most popular and widely used unix/linux job scheduler is Cron. Cron jobs are a great and reliable way to automate tasks and run scripts at regular intervals. In traditional unix/linux systems, you can just run
crontab -e and pop in
* * * * * bash /path/to/script.sh, or drop a file into
/etc/cron.d. However, with the advent of Docker and Kubernetes, setting up cron jobs is no trivial task. This article is meant to detail my experiences with Cron, and provide some examples on how to automate tasks in Docker and Kubernetes.
Docker + Cron
The use case is simple, but it took a surprising amount of research and tinkering before finding a cron solution that would suit my needs. Eventually, I found a great article that helped me get started.
The basic idea of this Docker image is that a user can place cron files into
Cronfiles, build a container, and run it via
docker run bambash/docker-cron. In the example cron file there is output redirection to stdout/stderr:
## output 'hello' to stdout/stderr
* * * * * root echo hello > /proc/1/fd/1 2>/proc/1/fd/2
One could also mount a volume containing cron files like this (used below in kubernetes config maps):
$ docker run -v /path/to/my/Cronfiles:/etc/cron.d bambash/docker-cron
Kubernetes + Docker + Cron
Running cron jobs in Kubernetes can be done a few different ways. Typically I use a variant of docker image above or leverage the built in cron engine. Which method to use depends on the scenario. I’ve found that if a small task needs to be executed at least once every 30 minutes — go with use case #1. The Kubernetes CronJob resource will spin up a new job pod based on the cron schedule. For frequent jobs, this can be a taxing process on the Kubernetes nodes. Especially smaller nodes.
Use Case #1
Because the image above will execute any cron files in
/etc/cron.d, we can leverage ConfigMaps and mount a volume into a cron pod.
First create the ConfigMap from a directory that contains cron files:
# kubectl create configmap cronfiles --from-file Cronfiles/
configmap "cronfiles" created
Then create the cron deployment:
# kubectl apply -f https://raw.githubusercontent.com/bambash/kube-cron-example/master/deployment.yaml
After a bit of time, you should see the output:
# kubectl logs -f cron-7cf5bd99b7-bcq8g
Thu Mar 29 04:42:02 UTC 2018 hello from kubernetes
Thu Mar 29 04:43:01 UTC 2018 hello from kubernetes
Thu Mar 29 04:44:01 UTC 2018 hello from kubernetes
Use Case #2
This solution is pretty fun, I use it for long running jobs like backups or batch processing. CronJobs will spawn a new Job pod based on the cron frequency.
- name: example-cronjob
- i am a cronjob
schedule: '*/5 * * * *'
Create the resource:
List jobs spawned:
# kubectl get jobs
NAME DESIRED SUCCESSFUL AGE
example-cronjob-1522301640 1 1 10m
example-cronjob-1522301700 1 1 5m
example-cronjob-1522301760 1 1 26s
List finished job pods:
# kubectl get pods -a
NAME READY STATUS RESTARTS AGE
example-cronjob-1522301700-4p68q 0/1 Completed 0 10m
example-cronjob-1522301760-x4nwd 0/1 Completed 0 5m
example-cronjob-1522301820-5qtcs 0/1 Completed 0 1m
Get logs from finished job pod:
# kubectl logs example-cronjob-1522301820-5qtcs
i am a cronjob
While Cron may be the traditional way of scheduling jobs in unix/linux, I believe it still has a place in the world of containers and automation. If anyone has other methods for handling cron jobs/scheduled tasks in Docker or Kubernetes, please share!
Thanks for reading!