A Beginner’s Guide to .NET’s HostBuilder: Part 4 — Configurations and Environments

9 min readOct 28, 2023

When .Build()-ing the host, the host builder will automatically read in settings from the following sources (in this order) to create an internal IConfiguration, used to represent the sum of the configs. If a setting is set by a subsequent source, then the setting is updated to the new value, so later sources can override earlier sources.

appsettings.json (iff this file is in the working directory)
appsettings.{Environment}.json (iff the environment is specified and this file is in the working directory)
.NET’s secret manager (iff enabled and the app is running on a developer’s machine)
Environment variables
Command line arguments

This part covers these configurations (AKA app settings) and environment features built into .NET’s HostBuilder. Here is the console app’s current working structure:

$ tree -I 'obj|bin'  # This will print the tree structure, ignore obj/ and bin/
.
├── appsettings.Development.json
├── appsettings.json
├── Driver.cs
├── HostBuilderDemo.csproj
├── Program.cs
└── Properties
    └── launchSettings.json

1 directory, 6 files

The series

Readers may need to refresh after clicking a section

appsettings*.json

The appsettings*.json files are the main way settings are supplied to the host builder. The worker or webapi templates will automatically create appsettings.json and appsettings.Development.json.

As mentioned earlier, appsettings*.json files can only be read in if they are in the working directory (or the path) when the host builder builds the host.

For demonstration purposes, the console app will take the startup message as a configuration. Start by making and retrieving Driver.Settings:

using Microsoft.Extensions.Options;

namespace HostBuilderDemo;

public class Driver
{
    private readonly Settings _settings;

    // Wrap the settings class in IOptions; this will be discussed later.
    public Driver(
        IOptions<Settings> settings)
    {
        Console.WriteLine($"Instantiating {nameof(Driver)}");
        _settings = settings.Value;
    }

    public Task RunAsync(
        CancellationToken cancellationToken)
    {
        // Use the startup message setting:
        Console.WriteLine(_settings.StartupMessage);
        return Task.CompletedTask;
    }

    // Added this settings class:
    public class Settings
    {
        public string StartupMessage { get; set; } = "";
    }
}

Next, bind the "Driver" section of the configuration to the Driver.Settings class in Program.cs:

using HostBuilderDemo;

IHost host = Host.CreateDefaultBuilder(args)
    // Use another overload's delegate to get the hostContext:
    .ConfigureServices((hostContext, services) =>
    {
        IConfiguration config = hostContext.Configuration;

        services.AddTransient<Driver>();
        // Bind the host builder's config's Driver section to the
        // Driver.Settings class:
        services
            .AddOptions<Driver.Settings>()
            .Bind(config.GetRequiredSection("Driver"));
    })
    .Build();

Console.WriteLine("Starting");
await host.Services.GetRequiredService<Driver>().RunAsync(default);

Finally, add section "Driver" to appsettings.json:

{
  // Added this section:
  "Driver": {
    "StartupMessage": "Hello from appsettings.json"
  },
  // Ignore the Logging section, this will be discussed in
  // the Logging and Serilog article.
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  }
}

Now the app can be ran with the following output:

$ dotnet run
Building...
Starting
Instantiating Driver
Hello from appsettings.json

Validating settings

There is a weakness with Driver.Settings: if the startup message is not supplied, or given a blank value, or the path is typo-ed, then the app will operate with sub-optimal settings. As such, it is highly recommended to use data annotations on settings classes to protect against trivial strings, negative integers, bad string patterns, etc.

These will run when the settings are retrieved from IOptions<T> (and related classes).

First, install package Microsoft.Extensions.Options.DataAnnotations.

Second, update Driver.cs, adding the Required annotation:

using System.ComponentModel.DataAnnotations;
using Microsoft.Extensions.Options;

namespace HostBuilderDemo;

public class Driver
{
    private readonly Settings _settings;

    public Driver(
        IOptions<Settings> settings)
    {
        Console.WriteLine($"Instantiating {nameof(Driver)}");
        _settings = settings.Value;
    }

    public Task RunAsync(
        CancellationToken cancellationToken)
    {
        Console.WriteLine(_settings.StartupMessage);
        return Task.CompletedTask;
    }

    public class Settings
    {
        // Add this annotation:
        // (When annotating a string, Required will ensure the string is not
        // null, blank, or whitespace).
        [Required]
        public string StartupMessage { get; set; } = "";
    }
}

Finally, update Program.cs to validate Driver.Settings on retrieval from IConfiguration (when instantiating Driver):

using HostBuilderDemo;

IHost host = Host.CreateDefaultBuilder(args)
    .ConfigureServices((hostContext, services) =>
    {
        IConfiguration config = hostContext.Configuration;

        services.AddTransient<Driver>();
        services
            .AddOptions<Driver.Settings>()
            .Bind(config.GetRequiredSection("Driver"))
            // Add these two lines:
            .ValidateDataAnnotations()
            .ValidateOnStart();
    })
    .Build();

Console.WriteLine("Starting");
await host.Services.GetRequiredService<Driver>().RunAsync(default);

Now update appsettings.json to the following to confirm that the validation works by making an invalid Driver.Settings.StartupMessage:

{
  "Driver": {
    // Update this setting so the property gets in invalid value.
    "StartupMessage": "      "
  },
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  }
}

Output:

$ dotnet run
Building...
Starting
Instantiating Driver
Unhandled exception. Microsoft.Extensions.Options.OptionsValidationException: DataAnnotation validation failed for 'Settings' members: 'StartupMessage' with the error: 'The StartupMessage field is required.'.
   at Microsoft.Extensions.Options.OptionsFactory`1.Create(String name)
   at Microsoft.Extensions.Options.UnnamedOptionsManager`1.get_Value()
   at HostBuilderDemo.Driver..ctor(IOptions`1 settings) in /home/sawyer/Code/HostBuilderDemo/Driver.cs:line 14
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Span`1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitDisposeCache(ServiceCallSite transientCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Program.<Main>$(String[] args) in /home/sawyer/Code/HostBuilderDemo/Program.cs:line 18
   at Program.<Main>(String[] args)

Environments and appsettings.Development.json

After appsettings.json is loaded by the host builder, the value from environment variable DOTNET_ENVIRONMENT (or ASPNETCORE_ENVIRONMENT for web apps) is retrieved, and then appsettings.{Environment}.json is loaded.

There are three environments built out-of-the-box: Production, Staging, and Development. Note that Production gets appsettings.json, and Production is the default environment (and to reiterate, appsettings.json is loaded first regardless of environment).

When running locally, Properties/launchSettings.json is used by the SDK, and this file can control a number of things, including environment variables:

{
  "profiles": {
    "HostBuilderDemo": {
      "commandName": "Project",
      "dotnetRunMessages": true,
      "environmentVariables": {
        "DOTNET_ENVIRONMENT": "Development"
      }
    }
  }
}

To observe environments, update the app in two places.

First, update the appsettings.Development.json (this will override apsettings.json’s value):

{
  "Driver": {
    "StartupMessage": "Hello from appsettings.Development.json"
  },
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  }
}

Second, the app can programmatically respond to the environment via hostContext.HostingEnvironment.Is{Environment}(). This is most commonly used by the web host builder to build Swagger pages when in Development because Swagger pages in Production can be a security issue. Update Program.cs to demo this functionality:

using HostBuilderDemo;

IHost host = Host.CreateDefaultBuilder(args)
    .ConfigureServices((hostContext, services) =>
    {
        IConfiguration config = hostContext.Configuration;

        // Add this paragraph:
        if (hostContext.HostingEnvironment.IsDevelopment())
        {
            Console.WriteLine("We're in dev");
        }

        services.AddTransient<Driver>();
        // Bind the host builder's config's Driver section to the
        // Driver.Settings class:
        services
            .AddOptions<Driver.Settings>()
            .Bind(config.GetRequiredSection("Driver"))
            .ValidateDataAnnotations()
            .ValidateOnStart();
    })
    .Build();

Console.WriteLine("Starting");
await host.Services.GetRequiredService<Driver>().RunAsync(default);

Output:

$ dotnet run
Building...
We're in dev
Starting
Instantiating Driver
Hello from appsettings.Development.json

Remove the new if statement from Program.cs and the Console.WriteLine("Starting"); near the bottom of Program.cs.

Development secret management

After the appsettings*.json file(s) have been loaded, if the code is running a developer’s machine, then user secrets are read from the .NET SDK’s secret manager. That documentation is solid, so this article will provide cliff notes.

Developers should place their secrets here to ensure that they do not accidentally, inevitably check-in secrets placed into an appsettings to version control.

First, ensure that the csproj is configured with the local secret manager:

$ dotnet user-secrets init

Next, create/set a user secret for Driver.Settings.StartupMessage to "Hello from user secrets". This will override appsettings.Development.json's value.

$ dotnet dotnet user-secrets set "Driver:StartupMessage" "Hello from user secrets"

Note the syntax for specifying the Driver's StartupMessage; while a json would have objects, this is the flattened syntax. In the flattened format, : (and __) are used to represent stepping into an object.

Output:

$ dotnet run
Building...
Instantiating Driver
Hello from user secrets

Environment variables

After the user secrets have been read in (if applicable), environment variables are read. These use the flattened syntax, and can be set in Properties/launchSettings.json for reproducibility between developers.

Create environment variable for Driver.Settings.StartupMessage in Properties/launchSettings.json (below). This will override the secret’s value. Note that when creating a true environment variable, Unix will not support the : delimiter; instead, __ must be used.

{
  "profiles": {
    "HostBuilderDemo": {
      "commandName": "Project",
      "dotnetRunMessages": true,
      "environmentVariables": {
        "DOTNET_ENVIRONMENT": "Development",
        "Driver:StartupMessage": "Hello from environment vars"
      }
    }
  }
}

Output:

$ dotnet run
Building...
Instantiating Driver
Hello from environment vars

Command line arguments

After environment variables are read in, command line arguments are read. These use the flattened syntax like user secrets and environment variables.

Run the app and set Driver.Settings.StartupMessage as a command line argument. This will override the environment variable’s value.

$ dotnet run --Driver:StartupMessage "Hello from CLI args"
Building...
Instantiating Driver
Hello from CLI args

Note that there are other syntax available when specifying command line arguments, and they are detailed here.

The “Intended” pattern for configs and environments

With all of this information in mind, here is my reading on the “intended” pattern for configs and environments.

Use settings classes, ideally with data annotations.
appsettings.json contains the production settings, and staging/development settings are overridden by the corresponding appsettings. When checked into source control, these files must not contain secrets.
When a developer works on a project, their next steps will depend on whether their workspace has development servers or not. If there are development servers, appsettings.Development.json will be used for that server and developers will override settings as needed via Properties/launchSettings.json. If there are not development servers, then appsettings.Development.json can be used for local settings. Either way, if they need to set secrets, they will use the built in secret manager.
When servers run a C# app, the app will be deployed with appsettings.json and the correct lower environment for the target server. The server will also have the DOTNET_ENVIRONMENT / ASPNETCORE_ENVIRONMENT configured appropriately so the host builder knows to read in the correct supplemental settings. If the app needs secrets, it is insecure for the secrets to live in plain text, be it any of the appsettings*.json, environment variables, or command line arguments. Ideally, the secrets are read from a secret manager that integrates with the host builder.

An alternate pattern for configs and environments

I have seen a few other patterns for using configs and environments, and I want to detail the alternate pattern I like the most. This pattern is characterized by not relying on the servers to correctly set DOTNET_ENVIRONMENT / ASPNETCORE_ENVIRONMENT and instead controlling the settings sent to the server when the code is deployed (ideally via a deployment pipeline).

I prefer this for a few reasons. First, the “intended” pattern is a mess of implicit overrides and what exactly appsettings.Development.json is used for. Second, it depends on servers’ environment variables being configured correctly and leaving yourself open to errors on that front. This way, the server explicitly gets the settings that are appropriate to that server, reducing error vectors. Additionally, it allows appsettings.Development.json to always be used for developer settings.

Use settings classes, ideally with data annotations.
appsettings.json has stub values (empty string, zero, etc) for environment-specific values and has the real values for environment-agnostic settings.
When a developer works on a project, they will maintain appsettings.Development.json as a full listing of their settings (continuing to use .NET’s secret manager as appropriate).
When deploying C# to a server, the deployment pipeline will inject the environment-specific values into the stubbed appsettings.json and the pipeline will not deploy appsettings.Development.json. If the app needs secrets, it is insecure for the secrets to live in plain text, be it any of the appsettings.json, environment variables, or command line arguments. Ideally, the secrets are read from a secret manager that integrates with the host builder.
If the app has environment-specific logic (such as building Swagger pages for a web API), the environment will need to be defined manually. First, add the key-value pair "Environment": "Development" to the root of appsettings.Development.json. Second, add the stub key-value pair "Environment": "" to the root of appsettings.json (and be sure to set this value when deploying, like the other stubbed values in appsettings.json). Here is the Program.cs that retrieves and configures that value:

using HostBuilderDemo;

IHost host = Host.CreateDefaultBuilder(args)
    .ConfigureServices((hostContext, services) =>
    {
        IConfiguration config = hostContext.Configuration;
        hostContext.HostingEnvironment.EnvironmentName = config["Environment"]
            ?? throw new InvalidOperationException("'Environment' was not defined in IConfiguration");

        services.AddTransient<Driver>();
    })
    .Build();

await host.Services.GetRequiredService<Driver>().RunAsync(default);

Addendum: working with IConfiguration directly (and ConnectionStrings)

It is possible to retrieve values directly from the IConfiguration. Additionally, there is a predefined, supported section in the root of the configuration called ConnectionStrings. For a demonstration of this functionality, consider the following files. Additionally, IConfiguration can be retrieved as a service via a service class’ constructor with no additional setup.

appsettings.Development.json:

{
  "MyName": "Sawyer",
  // ConnectionStrings is a built-in section with a specific retrieval function.
  "ConnectionStrings": {
    "BillingDb": "Server=localhost;Database=billing;Integrated Security=true;"
  },
  "Driver": {
    "StartupMessage": "Hello from appsettings.Development.json"
  },
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  }
}

Program.cs:

using HostBuilderDemo;

IHost host = Host.CreateDefaultBuilder(args)
    .ConfigureServices((hostContext, services) =>
    {
        IConfiguration config = hostContext.Configuration;
        Console.WriteLine(config["MyName"]);
        Console.WriteLine(config["Driver:StartupMessage"]);
        Console.WriteLine(config.GetConnectionString("BillingDb"));

        services.AddTransient<Driver>();
        services
            .AddOptions<Driver.Settings>()
            .Bind(config.GetRequiredSection("Driver"))
            .ValidateDataAnnotations()
            .ValidateOnStart();
    })
    .Build();

await host.Services.GetRequiredService<Driver>().RunAsync(default);

Addendum: different setting lifetimes

IOptions<T> is not the only way to bind and retrieve settings classes from IConfiguration: IOptionsSnapshot<T> and IOptionsMonitor<T> exist as well.

IOptions<T> uses the configuration that is captured at startup and it is a singleton.
IOptionsSnapshot<T> uses the snapshot of the configuration that occurs at the start of a new scope. This has a scoped lifetime.
IOptionsMonitor<T> monitors the configuration for changes as well as supporting notifications. If an app needs to retrieve updated values mid-process, this is the option to choose. These are singletons as well.

A Beginner’s Guide to .NET’s HostBuilder: Part 4 — Configurations and Environments

The series

Table of contents

appsettings*.json

Validating settings

Environment variables

Command line arguments

The “Intended” pattern for configs and environments

An alternate pattern for configs and environments

Addendum: working with IConfiguration directly (and ConnectionStrings)

Addendum: different setting lifetimes

Written by Sawyer Watts