Timezones in K8s Container

Zhimin Wen
Jul 30 · 3 min read

I am tracking some EOD job status to check if it starts late than the scheduled time, or overruns. Created as a Prometheus client program with Golang, the data are collected as a Prometheus gauge metric to be displayed in the Grafana dashboard using the table format.

Worked and tested perfectly on my laptop.

However, I had the typical problem that we commonly encountered. Yes, it worked on my laptop ;) It doesn’t work when I deployed into Kubernetes running as a container!

After some troubleshooting, I realized it's the default timezone in the container that causes the issue.

Verify

Launch a container, I am using Ubuntu image because of some LIBC dependency of my program.

By default, the time is UTC time. So when my program gets the current time it's actually the UTC time, which is 8 hours away from my local time. This was where the problem happened.

It’s noticed that in the default container image, the time zone data file is not installed.

Investigate

I can modify my program to use local time based on some timezone settings. Test it out with the following sample,

Compile it, copy it to the above container using the default image, run it

The reason is that the time zone data is not installed by default. Install the package of tzdata non interactively,

Now the file /usr/share/zoneinfo/Asia/Singapore is created, and the program is working fine,

We can import the tzdata package of golang to embed the zone data in the program. But a better approach is to leave this to the base OS, that is to install the tzdata package and set the TZ environment variable, now run the same program again,

The default time we got is the local time now.

Let’s fix the problem.

Fix

When building the container image, install the tzdata package non-interactively by inserting the following into the Dockerfile,

No change on the code.

But, simply define an environment variable in the K8s deployment YAML file as

Enjoy

The container app runs fine now.

Everything connected with Tech & Code. Follow to join our 500K+ monthly readers