Configuration & Automation in KCL

Introduction

Kusion Configuration Language (KCL) is an open source constraint-based record and functional language. KCL improves the writing of a large number of complex configurations through mature programming language technology and practice, and is committed to building better modularity, scalability and stability around configuration, simpler logic writing, fast automation and good ecological extensionality.

Prerequisite​

Install KCL:

Each release of KCL includes various OSes and architectures. These binary versions can be manually downloaded and installed from Github and add {install-location}/kclvm/bin to the environment PATH.

export PATH=$PATH:{install-location}/kclvm/bin

Configuration

The best way to learn a new language is to write a few small programs, and the same goes for configuring languages. We can write KCL programs just like writing configuration.

Here is a simple hello.k:

hello = "KCL"

Set the hello attribute to the "KCL" string. Then save the code to the hello.k file.

How to execute this program depends on the specific development environment, we first assume that the local macOS or Linux system has installed the kcl command (or enter the Docker environment test by docker run --rm -it kcl-lang/kusion bash) and then run the following command:

$ kcl hello.k
hello: KCL

The effect of command line execution is shown as follows:

The output is configuration data in YAML format. Although this program is simple, we can verify the basic usage of the development environment and the kcl command line by executing the KCL configuration program to the output.

2. A little more complicated configuration​

In addition to the common key-value pairs, common configuration data also has nested dictionary and list types, and the value basic type includes boolean and numeric types in addition to strings. Here’s a slightly more complex server.k configuration:

# This is a KCL document

title = "KCL Example"
owner = {
    name = "The KCL Authors"
    data = "2020-01-02T03:04:05"
}
database = {
    enabled = True
    ports = [8000, 8001, 8002]
    data = [["delta", "phi"], [3.14]]
    temp_targets = {cpu = 79.5, case = 72.0}
}
servers = [
    {ip = "10.0.0.1", role = "frontend"}
    {ip = "10.0.0.2", role = "backend"}
]

where # begins with a line comment. The value of owner is a dictionary. The value of the dictionary contains the content in the form of {}. The key-value inside the dictionary is similar to the hello = "KCL" example. database is another dictionary in which the value of the dictionary attribute appears boolean True, list [] and dictionary {}, in which the value of the numeric type also appears in the list and dictionary. The servers attribute is a list with dictionaries nested inside the list (dictionaries and lists, as well as the schema that will be discussed later, can be nested within each other).

The YAML output of this configuration is as follows:

$ kcl server.k 
title: KCL Example
owner:
  name: The KCL Authors
  data: '2020-01-02T03:04:05'
database:
  enabled: true
  ports:
  - 8000
  - 8001
  - 8002
  data:
  - - delta
    - phi
  - - 3.14
  temp_targets:
    cpu: 79.5
    case: 72.0
servers:
- ip: 10.0.0.1
  role: frontend
- ip: 10.0.0.2
  role: backend

3. Define the structure of the configuration using KCL schema​

The KCL provides abstract support for attributes with a fixed attribute structure and default value behavior through the schema syntax.

For example, the configuration of database in the above example is generally the default value. We can define a structure for the default configuration of the database:

schema DatabaseConfig:
    enabled: bool = True
    ports: [int] = [8000, 8001, 8002]
    data: [[str|float]] = [["delta", "phi"], [3.14]]
    temp_targets: {str: float} = {cpu = 79.5, case = 72.0}

enabled is a boolean type; ports is an integer list type; data is a list of lists, and the inner list elements are strings or floats; temp_targets is a dictionary type, and the attribute value of the dictionary is floating point type. And each attribute of DatabaseConfig defines a default value.

Then pass database = DatabaseConfig {} to generate a structure with the same attributes as the default values. We can also modify the default value:

database = DatabaseConfig {
    ports = [2020, 2021]
}

schema DatabaseConfig not only provides default values for attributes, but also adds type information to attributes. Therefore, if we accidentally writes the wrong attribute value type, KCL will give a friendly error prompt, such as the following example where ports is wrongly written as a floating point type:

database = DatabaseConfig {
    ports = [1.2, 1.3]
}

When executed, an error similar to the following will be generated (the displayed file path depends on the local environment):

$ kcl server.k 
KCL Compile Error[E2G22] : The type got is inconsistent with the type expected
    ---> File /path/to/server.k:8:2
    8 |    ports = [1.2, 1.3]
      5    ^  -> got [float(1.2)|float(1.3)]
    ---> File /path/to/server.k:3:2
    3 |    ports: [int] = [8000, 8001, 8002]
      5    ~  -> expect [int]
expect [int], got [float(1.2)|float(1.3)]

Similarly, we can encapsulate the attributes of the servers section with the following code:

schema ServerConfig:
    ip: str
    role: "frontend" | "backend"

servers = [
    ServerConfig {ip = "10.0.0.1", role = "frontend"}
    ServerConfig {ip = "10.0.0.2", role = "backend"}
]

The attribute ip of ServerConfig is a string type, and no default value is given. We must manually add the value of the ip attribute when generating the ServerConfig type attribute, otherwise the KCL will report a missing required attribute error. The role attribute is a "frontend" | "backend" enumerated string type.

In addition, schema can also combine check, mixin, optional attributes, inheritance and extension modules to achieve more complex configuration and policy data abstraction, details can be found at here.

Automation

KCL provides many automation-related capabilities, mainly including tools and multilingual APIs. Via package_identifier : key_identifier mode, KCL supports the indexing of any configured key value, thus completing the addition, deletion, modification and query of any key value. For example, the following figure shows that we can directly execute the following command to modify the image. The code diff before and after modification is also shown in the figure (Image Auto Updater).

In addition, the automation capability of KCL can be realized and integrated into CI/CD.

KCL allows us to directly modify the values in the configuration model through the KCL CLI -O|--overrides parameter. The parameter contains three parts e.g., pkg, identifier, attribute and override_value.

kcl main.k -O override_spec

• override_spec represents a unified representation of the configuration model fields and values that need to be modified

override_spec: [[pkgpath] ":"] identifier ("=" value | "-")

• pkgpath: Indicates the path of the package whose identifier needs to be modified, usually in the form of a.b.c. For the main package, pkgpath is expressed as __main__, which can be omitted. If omitted, it means the main package.
• identifier: Indicates the identifier that needs to modify the configuration, usually in the form of a.b.c.
• value: Indicates the value of the configuration that needs to be modified, which can be any legal KCL expression, such as number/string literal value, list/dict/schema expression, etc.
• =: means to modify the value of identifier.
• When the identifier exists, modify the value of the existing identifier to value.
• When identifier does not exist, add the identifier attribute and set its value to value.
• -: means to delete the identifier attribute.
• When the identifier exists, delete it directly.
• When the identifier does not exist, no modification is made to the configuration.

Note: When identifier appears multiple times, modify/delete all identifier values

Besides, we provide OverrideFile API to achieve the same capabilities. For details, refer to KCL APIs.

Examples​

Override Update Sample​

KCL code:

schema Person:
    name: str
    age: int

person = Person {
    name = "Alice"
    age = 18
}

The command is

kcl main.k -O :person.name=\"Bob\" -O :person.age=10

The output is

person:
  name: Bob
  age: 10

Besides, when we use KCL CLI -d argument, the KCL file will be modified to the following content at the same time

kcl main.k -O :person.name=\"Bob\" -O :person.age=10 -d

schema Person:
    name: str
    age: int

person = Person {
    name = "Bob"
    age = 10
}

Another more complicated example:

schema Person:
    name: str
    age: int
    ids?: [int]

person = Person {
    name = "Alice"
    age = 10
}

The command is

kcl main.k -O :person.ids=\[1,2\]

The output is

person:
  name: Alice
  age: 10
  ids:
  - 1
  - 2

Override Delete Sample​

KCL code:

schema Config:
    x?: int = 1
    y?: str = "s"
    
config = Config {
    x = 2
}

The command is

kcl main.k -O config.x-

The output is

config:
  x: 1
  y: s