Auto-Reload Configuration Problem in .NET Core applications on Kubernetes

Published in

Trendyol Tech

4 min readOct 5, 2020

In this post, we come up with the solution to a problem we had within the scope of Web Replatforming. As Trendyol Tech, we regard that the applications we develop are scalable and compatible with Cloud Native Design Principles. We work through and chew on the points where the technologies we use will not be compatible with the Cloud Native principles. This article will be about how we found a solution to one of these issues.

How did the problem arise?

A product that enables the applications running within Trendyol to make configuration management with a sidecar pattern was developed. My ex-teammate :) Onur Yilmaz explained this in detail in his post on this subject, “Configuration and Secret Management with Consul Template on Kubernetes”. We can say that this article evolved out of the trouble that occurred on the side of .NET Core when integrating this structure into the applications we developed.

What was the primary source of the problem?

First, let’s discuss how to do Kubernetes Auto-reload ConfigMap. They are included as environment variables when Kubernetes ConfigMap is created and added on pods. These environment variables do not change until you restart the pod. Therefore, you can introduce ConfigMaps into the pod as files on Kubernetes. Then, you can instruct your application to take the actions such as follow this file, reload the configs in case of a change. The FileConfigurationProvider and onReloadChange features are built-in in the .NET Core, but it comes to a head there in this feature.

When Kubernetes ConfigMap is updated, it always writes the change to a new file. The reason for this was not to affect the application if it was processing that file at that moment, but if it was constantly writing a new file, how would the application understand the new file this time? For this, there is a structure called Symbolic Link (Symlink) on Operating Systems. Indeed, it runs with the same logic as shortcut files. You set a target to this file, and this target starts to be read. When writing the ConfigMap to the file, Kubernetes changes this target and redirects to the new file. Thus, it does not interfere with the file being read and so prevents the application from having problems.

The structure of Kubernetes works very well and defensively. The problem starts on .NET Core. At the moment, there is no Symlink support on .NET Core. Accordingly, FileSystemWatcher doesn’t pay attention to other System.IO structures. The .NET Core application running on Kubernetes controls the CreationDate and LastModifiedDate of the file, and it does not need to read the file again as it does not see any changes. For this reason, Kubernetes Auto-Reload ConfigMap will not work in your .NET Core applications. You can do this by using Kubernetes APIs to get current configs. Here, you need to create an authorized Service Account and make your application access with these permissions.

So how did we solve this?

You can auto-reload ConfigMap with Kubernetes APIs, but the concept of Service Account here is a problem for us. Many issues, such as deployment or the creation of an independent control and security problems that may arise apart from that, are included in a new cluster. Our application shall be independent. It shall work even if you run it on a Windows server. It shall work everywhere since what we do is indeed File changes tracking.

In fact, we generate a simple solution. We control the file as an interval with the timer. We utilize the checksum method to detect file change. We produce and check the hash of the file with the hash calculation algorithm. In case of a change, ChangeToken notifies and informs the FileConfigurationProvider to read the file again, and then the file is read again. You can find this implementation in other documents in different ways.

First, let’s produce a custom ChangeToken and make a checksum with Timer inside. We can easily implement the IChangeToken interface.

Here, let’s extract the checksum function as an extension method and implement it as well.

Then implement IFileProvider and determine the issues such as which custom ChangeToken it will use, etc. Afterward, we will give this FileProvider to ConfigurationBuilder.

Let’s Finish Up

With this implementation, we have resolved the problem created by Kubernetes ConfigMap on Symlink file and .NET Core applications and thus, file tracking can be done easily. I hope Symlink Native support comes in new .NET versions and these processes continue as built-in.

Don’t forget to follow us on Social Media channels.