Automatically request and use Let’s Encrypt certificates in Dotnet Core

Let’s Encrypt is the “free, automated, open Certificate Authority” that wants every website to use encryption by default, by making it easy and — importantly — free. These are laudable goals and there is really no downside to using it. Except for the easy part… Unless you are lucky enough to use a hosting provider that configures these certificates for you, or use a software platform that has this built-in, then it might be easy. For a dotnet project running on Azure, I did not find an easy solution. Until Microsoft creates a checkbox for this in Azure (they really should), I’ll show how to request and install a free certificate at startup of an AspNet Core application yourself.

This is part of a series in my exploration of a modern web architecture, RROD. For the introduction, see here.

Acme and Dotnet

Luckily, the hard work of building a dotnet client library for the protocol that Let’s Encrypt uses, ACME (Automated Certificate Management Environment) has already been done. I found two implementations on nuget: Oocx.ACME and Certes. The first works, but it’s not actively maintained and doesn’t support the Pfx certificate format on dotnet core. Certes is cross platform, full featured and actively maintained.

I initially had some problems with Certes: I was running the request process in an Orleans Actor/Grain — which is nice because it makes sure only one request process can be started for a server farm — but it would just immediately hang. After some debugging I found out why: Certes uses the crypto library BouncyCastle, which uses a random number generator that uses the task scheduler as its entropy source: it starts a background task that increments a static integer in a tight loop (overflowing it) until the task scheduler stops the task, then it uses that integer as its next random number. This does not work well when running in an Actor system like Orleans, where every Actor/Grain runs in a single thread. New tasks are queued and delayed until the current task is finished, so the BouncyCastle random number generator puts itself in an infinite loop in Orleans. Once I understood the problem it was easy enough to work around: ultimately I decided to sidestep the whole problem by not running it in an Actor/Grain at all.

The certificate request now runs on the web front end and I decoupled the Certificate request process from Orleans. This way you can use this code in any dotnet core web project. The disadvantage is that for a large server pool some locking mechanism should be added: if many machines are started at the same time that will probably result in multiple certificate requests for the same domain.

The Acme protocol

The Acme protocol is a Web API that works like this:

• Register with the API using an email address. This address is not validated and is used to send a reminder email before the certificate expires (Let’s Encrypt certificates are valid for three months).
• Generate a certificate request for the domain (or domains) that the website runs on, and send the request to the API. This returns a challenge string consisting of two parts, separated by a dot.
• Now the Acme process validates your control of the domain by sending a request back to your site, to the the address: http://{yourdomain}/.well-known/acme-challenge/{first-part-of-challenge}. It is up to you to return the complete string, including the second part. This proves your control of this domain to Let’s Encrypt. If the certificate request has multiple domains in it, Let’s Encrypt validates every address in a separate validation request.
• You can poll the API for the certificate response. A few seconds after the validation was successful you can retrieve the response object containing the certificate authority signature from the API.
• With the request and response you can generate a Pfx file (a certificate file format that includes the key, and is password protected) and store it.

Dotnet Core and Https

With dotnet core and Kestrel, the usual way a website starts up with https looks something like this:

The UseKestrel() takes the certificate as an option and uses it immediately.

WebListener or Kestrel

Running under Windows, it’s also possible to do .UseWebListener() instead of .UseKestrel() which starts the website using HTTP.sys instead of Kestrel. In that case, the certificate needs to be registered in Windows using netsh commands, which can also be implemented with this project template by programmatically starting cmd.exe in a new Process and sending some netsh commands (always make sure to first remove prior bindings):

The advantage of WebListener is that it’s capable of binding to multiple urls on the same IP address (SNI), which can save costs for small websites. It’s also considered better tested, more secure and suitable for direct inbound traffic. Alas, everyone uses Kestrel anyway.

Startup

Whichever listener we use, to start with https, we always first need the certificate. And to get that certificate from Let’s Encrypt, we need to respond to an incoming request on plain http (port 80) on the same server. For which we need to start a listener. That means our startup has to be split in parts:

• first we start a listener on http, port 80, that listens on the /.well-known/acme-challenge/ url,
• then we start the Acme certificate request process, and pass the challenge response string to the listener so we can respond correctly to the challenge request,
• then we create the certificate and store it because we do not want to request a new certificate every time,
• only then can we do the real https startup, using the certificate.

Normally, host.Run() blocks execution until the server is stopped, but we can use host.Start() to just start a web listener and then continue to the second part of our startup.

In the RROD project, Program.cs is not the nice oneliner-startup of most dotnet core programs. I wanted a single configuration file for my website, no separate hosting.json and appsettings.json config files. And because the startup consists of several stages, and I did not want a Startup1 and Startup2, I merged the initialization that is usually in Startup.cs into one big Program.cs containing all the initialization code.

What happens in Program.cs is now:

• Read all configuration from json config files, environment and commandline sources
• Initialize the connection to the Orleans Server
• Parse the urls we want from the configuration and filter the secure ones
• If there are secure urls, we start a WebHost on port 80 to identify our webserver with Let’s Encrypt and handle the Acme Challenge. First we configure the AcmeCertificateManager service, that contains the function that we’ll call later to request the certificate and communicate the challenge response to the middleware. This manager takes an option object containing four delegates:
  1) Get a saved challenge response. The challenge response is simply a string that must be returned based on the challenge string
  2) Save a challenge response for a short duration (usually in a Cache)
  3) Store a certificate in PFX format (usually in a Database) The Pfx is always password protected.
  4) Retrieve a stored PFX certificate
• In the RROD code I’m using some Orleans Grain classes to store the certificate and challenge string. More on this in a future post. You could use any other technology, even a static dictionary will work when running on a single server.

You can see how everything works in the AcmeCertificateManager code.

• This listener then sets up the request pipeline with a single middleware: app.UseAcmeResponse() This middleware resolves the AcmeCertificateManager, and uses the GetChallengeResponse delegate to find correct responses to return when a challenge comes in.

Initialization, Part two

With the acme listener running we can start the real WebHostBuilder. In this builder, the website is configured using the normal middleware like Mvc, IdentityServer4 and JavascriptServices. This is all normal stuff.

If there are secure urls, the CertificateManager is resolved inside UseKestrel() and GetCertificate() is called. That will retrieve and decrypt a stored certificate, if none is found or the one found will expire within 14 days, it will request a new one from Let’s Encrypt and store it. (Let’s Encrypt certificates are valid for three months only so you need to make sure the service restarts every week or so).

Then, finally after some 458 lines of initialization, we get to the point:

host.Run();

That will kick of everything. If you have specified a https url in the urls parameter in appconfig.json it will request a new certificate on startup, save it and immediately start running on the https address.

Testing it

To see all this in action, clone the project and first make sure it runs without https. By default, it is configured to run on http://localhost:5000. The solution is configured to not use SQL Server, it uses Azure Storage, so you first need to install and start the Azure Storage Emulator (part of the Azure Workload in VS 2017 setup) or configure a connection string to a real Azure Storage Account (Orleans supports several other, non-Microsoft storage providers too). In Visual Studio, configure the solution to startup “Multiple Projects” and select OrleansHost and Webapp. Make sure Webapp starts as an executable, outside of IIS / IIS Express. Webapp will try to connect to OrleansHost, which will fail while it is not started completely, but Webapp will retry until it can connect. The errors in the console look scary but they can be ignored.

If you use VSCode, first start OrleansHost in the debugger, then add Webapp (you might need to configure dotnet core compilation and both projects in launch settings).

After you know it works on port 5000, you can try a https address. Obviously this needs to run on a public address for Let’s Encrypt to be able to validate it. You can use Azure or any other provider, but for testing it’s easier to use ngrok. Install it if you haven’t yet, and run:

ngrok http 80

Ngrok

You’ll see a screen like this one. Ngrok is a proxy that sends traffic from a public address to another port on the local computer. It proxies both http and https, but in this case we only need/use the http part. Copy the server address (in my case 06478ca8.ngrok.io) and now open your hosts file to alias that address to localhost:

This makes sure that locally on your computer, the ngrok address resolves to localhost, so the browser simply will connect to the Webapp project being debugged locally. Now configure that address for our Webapp, in appsettings.json:

We also need to configure the PfxPassword and email to use. I am using the “User Secrets” feature in Dotnet core: in Visual Studio 2017, right-click the Webapp project, select “Manage User Secrets” and put your secrets here. VS Code users can put this file manually in their home folder (the location is OS dependent, the subfolder name is configured in the csproj file). During development, the values in here will be used, outside of git source control. In production, you could put these values in an environment variable (using double underscores for the section, i.e.: AcmeSettings__EmailAddress).

The secrets.json file should look like this:

If you have done everything correctly, you should now be able to see everything together. Start OrleansHost and Webapp, then open the site in a fresh browser window.

In the console window, you should see the validation request from Let’s Encrypt being logged. If you open the site, you should get no security warning and the certificate should be a fresh new Let’s Encrypt certificate, not one belonging to Ngrok. You can check the Certificate in Chrome using F12 developer tools, under Security:

That’s it!