Introducing Stratio’s Vault Library: Simplifying Secrets Management in .NET using Hashicorp Vault

Empowering the .NET Community with an Open-Source Solution for secrets handling

In today’s digital landscape, businesses cannot compromise on the security of their sensitive information. As applications grow more complex, managing secrets, such as API keys, database credentials, other confidential data, or even special configurations, becomes a critical aspect of software development.

To address this challenge, Stratio developed the Stratio Vault Library, a .NET library designed to streamline secrets management for applications interfacing with Vault.

This library was initially aimed at improving and simplifying the life of our developers, as well as further increasing the security of our solutions.

After the development and internal adoption of this library, we believe we have achieved the initial objectives and decided it would make sense to make it widely available for the world to use and improve.

Streamlining Secrets Management

The Stratio Vault Library serves as a convenient layer on top of the well-established VaultSharp Library, acting as an extension of Microsoft’s Configuration and Hosting libraries, specifically tailored for applications written in .NET.

It aims to simplify the process of using secrets within .NET solutions by leveraging the appsettings.json file and Vault’s secret management software.

By utilizing the Stratio Vault Library, developers can seamlessly integrate their applications with secrets or configurations fetched from Vault without the need for extensive code modifications or manual configuration changes.

In order to adopt this, extension all you need to do is call .useVault from your HostBuilder: this will connect your application to Vault using vault configuration in your environment. From there, you can use secrets in your app settings files by using the placeholders defined by the library.

If you’re not here to read a wall of text and want to jump right into action, just follow the next link and you’ll get right to the repository where you can find the Stratio Vault Lib: https://github.com/stratio-automotive/Stratio.Extensions.Configuration.Vault.

This streamlined approach significantly reduces the complexity associated with secrets management, enabling developers to focus on building robust and secure applications.

How does it work?

Let’s suppose you are developing an API in .NET that connects to multiple SQL Server databases. In order to connect to the databases, you’ll need to have some kind of access credentials set up somewhere. Typically in a .NET project this information can be stored in the appsettings.json file. Take the example below:

{
  ...
  "Database": {
    "host": "database.myorg.com",
    "databases": {
      "client1": "client1Database",
      "client2": "client2Database",
    },
    "user": "apiuser"
    "password": "mypassword"
  }
  ...
}

For simplicity, let’s assume that the two databases in SQL Server share the same host, username and password.

As you’ve probably already guessed, this is not only an extremely unsecure way to store your credentials, as they’re stored in clear text in an accessible text file, but they’re also pretty much static when you think about a typical SDLC, constantly pushing right through various environments.

But what if there were a simpler way to achieve security and flexibility in this context? You’re probably looking to replace that sensitive information with variables, right? But where will you store the variables? And how can you assign values to those variables?

The storing of sensitive information is hardly a new topic of discussion among developers. Recent conferences and workshops have praised the efficiency of solutions like Vault or AWS Secrets Manager.

We wanted to give these tools a try and, for us, the choice was obvious, we needed a solution that wouldn’t tie us to a cloud provider, and we also wanted to have multiple installations of this tool, specifically on premises. Therefore, we went for Vault and started our quest to improve the security of our sensitive data.

Vault servers were set up, properly configured and tested. We managed to access secrets from Vault securely from our Kubernetes Clusters and CI Pipelines, but then we got to the stage of integrating these secrets into our .NET solutions.

We expected there would be some kind of library simple enough to read Vault secrets into our .NET solutions. There wasn’t. We’ve searched everywhere and found nothing that would even come close to making us raise our eyebrows and wonder if it might be it.

We even went as far as doing something unthinkable: we went onto the second page of Google Search results. The chills, the heresy, the capital sin!

Now, in all seriousness, there isn’t much out there that would simplify the task of getting our .NET solution to read Vault secrets. There are some alternatives apart from the widely used VaultSharp Library, however, they lack customization and flexibility, in that they have a very hard structure of Vault secrets organization.

The available authentication methods may be too few, or they’re simply too basic and/or very cumbersome to set up. That’s when we understood that we would have never been able to convince our Developers to adopt any of these libraries since the integration was way too complex, required too many changes, and didn’t meet our requirements.

That’s when we got the idea of adding a layer of abstraction on top of the library that was closer to Vault, which was indeed the VaultSharp Library. And just like that, the Stratio Vault Library started to take shape.

The Stratio Vault Library is indeed an abstraction layer on top of VaultSharp, but it’s set up in a way that makes it a lot easier to work with it. It’s nothing more than a NuGet package, with which you’re hopefully already familiar.

After including the NuGet package, it’s just a matter of calling .useVault() from your HostBuilder and, taking into account the previous example, customising your appsettings.json file as follows:

{
  ...
  "Database": {
    "host": "{% vault_secret sqlserver:host %}",
    "databases": "{% vault_dict sqlserver/databases %}",
    "user": "{% vault_secret sqlserver:username %}"
    "password": "{% vault_secret sqlserver:password %}"
  },
  "Vault": {
    "vaultAddress": "https://myvaultaddress.com:8200",
    "mountPoint": "myorg/prod",
    
    # For authentication via AppRole
    "approleAuthName": "approle",
    "roleIdPath": "PATH/TO/approle.role_id",
    "secretIdPath": "PATH/TO/approle.secret_id",
    
    # For authentication via Kubernetes Role
    "kubernetesAuthName": "kubernetes",
    "kubernetesSaRoleName": "kubernetes-role",
    "kubernetesSaTokenPath": "/var/run/secrets/kubernetes.io/serviceaccount/token"
  },
  ...
} 

As you can see, the secrets got replaced by placeholders and no more secrets are exposed on your repositories or your servers. Success!

You might have also noticed that there’s a new section within the Vault configurations. Here, you’ll configure how the library should connect to Vault. Currently, only Approle and Kubernetes methods are actively supported.

So, in short, this is how the library works:

1. Your appsettings.json file gets loaded;
2. The library processes the file in search for specific placeholders;
3. For each placeholder it finds, the library tries to load the secret from the configured Vault server and, if found, sets the configuration with the secret;
4. After all placeholders get replaced, the solution proceeds to work as intended.

If you need more information, the library comes with comprehensive documentation and examples, making it easy for developers to integrate and utilize its capabilities.

Key Features and Benefits

1. Seamless Integration: The Stratio Vault Library seamlessly integrates with .NET applications, providing a simple interface to interact with Vault. Developers can effortlessly use secrets without needing to actually type clear text secrets in the appsettings.json file.
2. Simplified Configuration: With less than a handful of configuration string patterns, managing secrets becomes a breeze. Developers can define secrets in the appsettings.json file, allowing for centralized configuration management and easy deployment across various environments.
3. Enhanced Security: The library promotes security best practices by securely retrieving and storing secrets from the Vault server. By separating sensitive information from the application code, the risk of accidental exposure or unauthorized access is significantly reduced.
4. Extensibility and Customization: As an open-source project, the Stratio Vault Library empowers developers to extend its functionality to meet their unique requirements. With community contributions, the library can evolve and adapt to the evolving landscape of secrets management, making it a future-proof solution.

Although we believe we’ve achieved most of our objectives with this library, we’re not done! We are committed to further improving this library (hopefully with a little help from the community) and adding other features, the most impactful being the hot reload of Vault secrets.

There’s nothing worse than having to restart all your containers when you change the credentials of your database engine. So, introducing a hot reload of secrets is one of our top priorities, which we believe will be critical to ensure a more meaningful and impactful adoption of your workloads.

Open-Source for Collaborative Innovation

Recognising the importance of collaboration and community-driven development, Stratio has made the Vault Library open-source. This decision ensures that developers from all backgrounds can contribute, enhance, and tailor the library to suit their specific needs.

Embracing the open-source ethos, we invite the .NET community to participate in this project.

Conclusion

As secrets management grows in importance, the Stratio Vault Library offers a valuable solution to simplify the integration of .NET applications with Vault.

By providing a user-friendly layer on top of the VaultSharp Library, it empowers developers to streamline secrets management, enhance security, and focus on delivering high-quality software.

By open-sourcing this library, Stratio invites the .NET community to contribute and collaborate, fostering innovation and collectively improving secrets management within the ecosystem.

Together, we can shape the future of secure application development and make secrets management a seamless experience for developers worldwide.

Join us in embracing the power of the Stratio Vault Library and let’s build a stronger, more secure .NET ecosystem together!