Apr 19, 2018
Accessing private NuGet feeds from a Docker Compose build on Visual Studio Team Services
Things are coming together in my exploration of Visual Studio Team Services and Docker Compose. This is going to be little journey into Visual Studio Team Services, private NuGet feeds and Docker Compose builds on a Hosted Linux agent.
I have been exploring VSTS for the past few months, especially for building Docker images. One thing I ran into pretty fast is the need restore NuGet packages from private feeds using the hosted agents.
Alongside my exploration of Visual Studio Team Services my team have started using MyGet and the Enterprise offering to manage our NuGet feeds to get a more granular control. So it’s time to figure out how to access those private feeds from a Docker build inside Visual Studio Team Services.
Visual Studio Team Services comes with a huge number of built in tasks including NuGet tasks for restoring and packaging etc. These allow you to store the credentials for feeds in the “Service endpoints” configuration of your project. This is a great feature, but, as far as I can tell, only works on the Windows agents and for regular non-docker builds.
What I needed was a way to provide the credentials for our private MyGet feeds in an easy and secure way. Storing the credentials in a NuGet.config file in the source repository is a bad idea, you don’t want your credentials stored there. I ended up with a quite simple solution and a one line bash command.
First up, secure variables
First of all you need to store the credentials in a secure way. Fortunately VSTS lets you define variables in a library and share those across build definitions in your project. It also lets you mark a value as “secure” by toggling a small padlock icon. It will not be visible in the UI from that point on. I believe it’s even possible to get the variables from Azure Key Vault, but I have not tried that yet.
These variables can be linked in the build definitions under “Variables > Variable groups” making them accessible to tasks during build.
Creating a NuGet.config file on demand
For .NET Core, that we base most of our newer services on, to be able to restore packages it needs a NuGet.config file, with the feeds and credentials in it, so we need to create one. I experimented with using Powershell to create the NuGet.config file and a custom NuGet task executing the nuget sources add command to add the private feed source. But that does not work on the Hosted Linux agent that I’m using. The great thing about nuget sources add is the ability to have the password stored encrypted in the file, but the encryption is tied to the user executing the command and won’t work once the NuGet.config file is copied into the container.
I really want an easy solution that is maintainable and still offers a practical amount of security and I feel we get that with the secure variables from the library. VSTS will ensure that whenever you are using a secure variable it will not show up in logs, it’s replaced with ********.
So I ended up using the “Shell script/bash” task that comes with VSTS. By default it’s set to use version 2.* of the task, but that only supports executing a script file, and not a simple inline script that I want. So be sure to change that to version 3.* (It is currently in preview, but for this it worked without issue). I use echo to write a string containing some NuGet configuration XML to a NuGet.config file. The file will end up something like this.
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="MyFeed" value="https://myfeed" />
</packageSources>
<packageSourceCredentials>
<MyFeed>
<add key="Username" value="$(NuGetUsername)" />
<add key="ClearTextPassword" value="$(NuGetPassword)" />
</MyFeed>
</packageSourceCredentials>
</configuration>The full command is:
echo '<?xml version="1.0" encoding="utf-8"?><configuration><packageSources><add key="MyFeed" value="https://myfeed" /></packageSources><packageSourceCredentials><MyFeed><add key="Username" value="$(NuGetUsername)" /><add key="ClearTextPassword" value="$(NuGetPassword)" /></MyFeed></packageSourceCredentials></configuration>' > NuGet.configThe only “magic” here is the substitution of variables done using $(NuGetUsername) and $(NuGetPassword). It just works.
With the NuGet.config now file available we can include that in the files we are copying into our container during a Docker build. The .NET Core build will then use that so it can restore the required packages from the private feeds.
Be aware that the NuGet.config file will be part of the image, so use multi stage docker builds and ensure that only your build output, not the NuGet.config, is copied to the image you end up pushing to the registry.
Another way I have seen the problem with NuGet feed credentials solved is by providing them in Dockerfile environment variables, but that is probably not such a good idea as the credentials might also be part of the image in that case. Keep that in mind.
The solution that I ended up with provides a fairly secure way to provide the credentials, they will not be readily visible in the UI or logs. Of course a malicious Dockerfile will be able to extract the credentials, but if your source control have been compromised to allow an attacker to change a Dockerfile, well, shit!
If you have other suggestions for working with private NuGet feeds or secrets in general, when building Docker images feel free to leave a comment.
On another topic, the last few weeks in Denmark the weather has been quite incredible as oppose to a normal April and I have spent a lot of great hours on my mountain bike. I can’t wait for summer!
