Mastering CoreDNS Configuration: Parsing the Corefile with Go Reflection

Our global DNS gateways, built on the robust CoreDNS platform, serve as the bedrock of our comprehensive security product. These gateways offer a wide range of features, including content filtering, secure blocking, and routing decisions for our Jamf Connect ZTNA solution.

CoreDNS is a highly adaptable and modular DNS server written in Go. It employs a plugin architecture to provide a wide range of features and capabilities, with each plugin configured using the Corefile. However, the current parsing mechanism used by CoreDNS can be inefficient, verbose, and error-prone, especially when dealing with nested blocks. To address these challenges, we developed a new parsing solution leveraging Go reflection, which promises to be less error-prone, more user-friendly and more efficient in terms of code, providing a straightforward and streamlined method for parsing the Corefile in CoreDNS.

The Corefile

The Corefile is a configuration file used by CoreDNS that defines the behavior of the DNS server. It is a text file that is typically located in the same directory as the CoreDNS binary. The Corefile allows users to configure DNS zones, forwarding rules and plugins for handling DNS requests.

. {
    hosts /etc/coredns/hosts
    forward . 8.8.8.8 9.9.9.9
    log
    errors
}

The Corefile is structured as a series of blocks, each of which represents a different DNS zone or configuration directive. Blocks are defined using curly braces and can be nested to create more complex configurations. Each block is identified by a domain name, which can be specified either as a string literal or a regular expression. Directives are specified within each block using a keyword followed by any relevant arguments and can be ordered arbitrarily.

For more information about the Corefile, see the documentation.

Initializing the plugin

When a plugin is initialized in CoreDNS, it must parse its configuration from the Corefile to determine how to handle DNS requests. To accomplish this, CoreDNS uses a controller called Caddy, which provides a powerful abstraction layer for managing plugins and their configurations.

The Caddy controller is responsible for parsing each plugin configuration from the Corefile. It uses a lexer, a type of tokenizer, to tokenize the Corefile, recognizing each symbol and grouping them into meaningful tokens. The lexer is programmed to recognize specific types of tokens such as keywords, values, and curly braces, and to differentiate between them based on their position in the Corefile.

For each plugin block, the Caddy controller creates a new instance of the corresponding plugin and calls its Setup() method to parse its configuration options. The Caddy controller then passes the configuration options to the plugin, allowing it to configure itself appropriately.

Configuring the plugin

Now we are getting to the problematic part. The Caddy controller knows the position of each token in the Corefile and provides the plugin with all the necessary tokens according to their position. However, there is no automated way to parse these values into a structured format other than manually looping through the tokens and calling the controller’s Next() function. Furthermore, there is no built-in way to check if the number of arguments provided by the user matches the expected number of arguments for a given plugin. As a result, we have to manually check the number of remaining arguments.

When dealing with nested blocks in the Corefile, the current approach to parsing becomes even more problematic. To handle nested blocks, we need to call the controller’s NextBlock() function and manually loop through all the elements of the block that are returned. This function can be used as the condition of a for loop to load the next token as long as it opens a block or is already in a block.

On every cycle of the loop, we need to call the Val() function to get the actual value of the element. We then require a big switch statement where we include all the keywords that can be used within a block. For every case in this switch, we would need to implement custom parsing of the specific element, which in most cases includes further looping using the controller, as additional nested blocks are not supported by the NextBlock() function.

In addition to the aforementioned difficulties, the process of parsing needs to be covered by unit tests to ensure the correct functionality of the parser and detect any bugs that might occur during development. This process involves a large file that accepts a string as an input, which is the Corefile configuration. The setup method is then called to check if the outcome matches the expected results. This requires creating a separate test case for every possible combination, whether it is right or wrong, resulting in a massive file full of test cases and string inputs for each of the plugins.

Our approach using Go reflection

While examining the existing parsing process, we identified various challenges, such as the code being difficult to maintain and prone to errors. We also observed a lot of repetitive for loops and unit tests that resulted in code cluttering. To tackle these issues, we decided to create a centralized solution that could be shared among all the plugins, streamlining the parsing process and making it more efficient.

By doing so, we aimed to eliminate the need for duplicating code and provide a more maintainable and scalable solution for handling configurations. In addition to being more efficient, we also wanted a solution that could dynamically assign values to variables with the correct data type, making it easier to use.

To accomplish this task, we employed two useful facets of the Go language: struct tags and reflection.

Struct tags

Struct tags are a potent feature of the Go programming language that provides a way to attach metadata to the fields of a struct. This metadata can be used to annotate fields with additional information such as field names, validation rules and formatting instructions. Struct tags are defined as a string literal placed after the type in a struct field declaration and are processed at runtime by the Go reflect package. This allows developers to define custom behavior for their types without needing to write boilerplate code, making it a powerful tool for creating reusable and flexible code.

Reflection in Go

Go reflection is a compelling feature that allows a program to inspect and manipulate its own code and data structures at runtime. It enables the program to examine the type and properties of objects, even those that are not known at compile time. Reflection is widely used in Go for various purposes such as serialization, deserialization and creating dynamic data structures. It is a powerful tool for creating generic algorithms and making code more reusable and flexible.

Unified and dynamic parsing solution

We devised a comprehensive solution that involved creating a dedicated package, called corefile, which houses all the necessary parsing logic. With this approach, plugins need to only import the package and invoke its Parse(c *caddy.Controller, v any) error method. The v argument is a pointer to a struct that represents the desired configuration, to which the parsing results will be automatically mapped.

Let’s consider a plugin with this Corefile configuration:

example_plugin argument {
  ip_address 192.168.0.1
  hostname localhost
  port 8080
  nested_config {
    id medium123 
    name radovan
    duration 10s
  }
}

To facilitate the parsing of such configurations into a structured format, we need to define corresponding Go structures,

type ExamplePluginConfig struct {
  Arguments           []string
  IPAddress           net.IP       `cf:"ip_address" check:"nonempty"`
  Hostname            string       `cf:"hostname" default:"host"`
  Port                int          `cf:"port" check:"nonempty,lte(8080)"`
  NestedConfiguration NestedConfig `cf:"nested_config"`
}

type NestedConfig struct {
  ID       string        `cf:"id" check:"nonempty"`
  Name     string        `cf:"name" check:"nonempty,oneOf(radovan|babic)"`
  Duration time.Duration `cf:"duration" default:"10s"`
}

followed by invoking the Parse() function during the plugin initialization process, as mentioned earlier, within the Setup() function.

func setup(c *caddy.Controller) error {
  var cfg ExamplePluginConfig
  if err := corefile.Parse(c, &cfg); err != nil {
    return err
  }

  // PLUGIN INITIALIZATION CODE

  return nil
}

After these steps, the configuration is thoroughly parsed into variables with their respective data types, while also having custom checks and default values applied to them.

If you’re interested in diving deeper into the implementation details, feel free to check out the accompanying Git repository. Happy coding!

The power of reflection

Now, let’s take a closer look at the inner workings. Our solution is based on the same concept as described earlier, which is to utilize the Caddy controller to navigate through the Corefile. We still rely on the controller’s Val(), Next(), and RemainingArgs() methods to process the configuration values.

At the heart of the reflect package is the reflect.Value type, which represents a value of any type, and provides a wide range of methods for inspecting and manipulating values at runtime. To take advantage of these capabilities, we call the reflect.ValueOf(i any).Elem() function with a pointer to the structure we want to parse the configuration into as the argument. This returns a reflect.Value type that represents the struct itself, rather than the pointer to the struct. By calling this function, we can then detect the type of each field in the struct, allowing us to assign the appropriate values to the appropriate data types while also applying default values using the struct tags mentioned earlier.

Our approach eliminates the need for numerous for loop calls, as parsing of the nested structures is handled by recursively calling the parsing method. This allows for cleaner and more concise code, while also providing a more flexible and robust solution for parsing configuration files.

Finding the fields

The first step is calling the Type() function on the reflect.Value object we are working with. This function returns a reflect.Type object that represents the type of the value. After that, we call the Kind() method on this result. This method returns a reflect.Kind value that represents the specific kind of the type.

If the returned Kind is a pointer, we need to check if the memory was allocated. If not, we allocate memory according to the type the pointer points to, using the reflect.New() method. If the returned Kind is a struct, we will recursively call our custom parsing method to handle the nested structure. Finally, if it is a regular field with some arguments, we will assign them to the structure.

We utilize the struct tags mentioned earlier to find the appropriate field of a structure. We use the NumField() method of the reflect.Type value to get the number of fields in the struct. We then loop over this number using a for loop and call the Field() method. Finally, we access the field's Tag property and call the Lookup() method to get the tag value we are looking for. If we find the appropriate field, we return its reflect.Value representation. An empty value is returned if we don't find the appropriate field.

Assigning the values

Once we find the appropriate field to assign the value into, we need to parse the input value and store it in the struct element. To retrieve the input arguments, we use the RemainingArgs() method. At this point, we have two values: the target reflect.Value and the input value as a string. We then check the Kind() of the target value to determine its type and parse the string input accordingly using Go's built-in conversion functions. Finally, we store the parsed value into the struct element using the Set() method of the reflect.Value type.

Plugin arguments

By including a field named Arguments in the structure, the parser identifies that the plugin arguments need to be parsed. This field, defined as a slice of the corresponding data type, serves as a destination for the parsed plugin arguments. For example, if the Arguments field is declared as []string, each plugin argument will be parsed as a string and assigned to this field.

Default values

By harnessing the power of struct tags and reflection, we can effortlessly assign default values to each element of the structure. To specify a default value, we use the struct tag format default:"value". If a custom default value is not provided, we use the zero value of the corresponding type. However, when a custom default value is specified, we extract its string representation and parse it using the same approach described earlier.

Elements validation

In addition to assigning default values, our solution also supports specifying custom checks for struct elements. We provide several built-in checks to validate the values.

The nonempty check ensures that the specified field has a non-zero value. This is useful for enforcing mandatory fields.

With the oneOf check, users can specify multiple allowed values, and the check ensures that the field contains one of these values. This provides flexibility in validating against a predefined set of options.

Furthermore, we offer simple mathematical checks such as lt (less than), lte (less than or equal), gt (greater than), and gte (greater than or equal). These checks allow users to define numeric constraints for the field's value, ensuring it meets specific requirements.

Advanced custom checks and initialization

In certain cases, simple struct tags may not suffice for performing complex checks or initialization on the parsed structure. To address this, we introduced two interfaces: CustomChecker and Initializer.

The CustomChecker interface allows for the implementation of more sophisticated checks. By implementing the Check() error method, custom validation logic can be performed on the parsed structure. This method can be implemented as a pointer or value receiver.

On the other hand, the Initializer interface facilitates custom initialization. The Init() error method allows us to perform more intricate initialization operations. As this method can directly modify the structure fields, it has to be implemented as a pointer receiver.

When either of these interfaces is implemented, the parser automatically invokes the respective method at the end of the parsing process.

Conclusion

We aimed to simplify the parsing process by leveraging the power of Go reflection. By doing so, we eliminated the need for multiple for loops and reduced the number of unit tests required. Our approach introduced a single parsing logic that is stored in one place and reused among all the plugins. This allowed us to create a more efficient and maintainable codebase. The use of Go reflection made it possible to dynamically access and manipulate data structures, which greatly simplified the parsing of configuration files. Our approach has proven to be a more effective and scalable solution for handling configuration parsing using the Caddy controller.

PS: the gopher image in this article is Creative Commons Attribution 4.0 licensed and was created by Renee French.