Christopher Laine

Follow

Jul 14, 2019 · 8 min read

In my last post, I took you through an introduction to ASP.NET Core health checks in your asp.net core microservices.

This time around, I thought I’d pick up where I left off with a more concrete example. I received a number of requests for more detail on how to actually implement health checks in a real-world scenario, so I thought I’d build an example so you can see health checks in action.

As before, I’ve whipped up a GitHub repo so you can see the code for yourself.

Note: In order to get the most out of the examples, you will need to be running Docker, as the example is a self-contained example with a couple Docker containers included. Visual Studio 2017/2019 and VS Code support Docker and Docker Compose, so you likely just need to get Docker running on your workstation. I’d explain how, but that’s a whole other kettle of fish.

Health Checks for our microservice: Why do we need them?

Let’s take a sec and analyse why we need health checks. We have an application / microservice, and this has x number of dependencies which it requires in order to be able to do its job. These could come in any number of forms:

• A data store of some kind, be it sql, nosql, what have you. Our microservice needs to be able to access this data store in order to read and write data.
• A file system or cloud storage Your microservice needs to be able to read / write files to disk.
• A cache server Clearly, a cache can make or break a system’s performance, so its best that our microservice has access to it.
• An OAuth2 / OpenId system Your microservice may very well depend on Identity and Access Control to do its job properly.

In addition, you might want to hustle up a nice dashboard for you and yours which shows the health of your microservices. This is handy dandy. At a glance you could see who is running happily, and who is in need of some TLC.

Now, with this in mind, if any one of the above resources goes offline, or if our microservice can’t access them, then our microservice is unhealthy (or maybe just degraded, but you know what I’m saying). We want our health checks to make it possible for our microservice to check in with its requisite resources and let us know that everything is hunky dory, or it is in trouble and needs our help.

I want to give big props to the team over on the AspNetCore.HealthChecks GitHub project for their amazing work. They make health checks a cakewalk.

For more information on what pre-built health checks they have, and how to use them, read over the wiki. That should get you started.

What will we be building?

Our imaginary microservice will need the following dependencies:

• Access to its MySql database If our microservice can’t reach the database, we will consider it “Unhealthy”
• Access to its Redis Cache We’ll go with the supposition that the cache is to improve performance, not critical to our service’s success, so if this is missing, we’ll consider our microservice “Degraded”.
• A custom health check all its own Let’s say you have some internal resource you need to reach, like some kind of license key file or a directory which your microservice should be able to access. Let’s assume if this is missing, you want the microservice to be “Unhealthy”.

A quick note on code samples

I won’t be adding all the code you need to run through this, but only those pieces which are related to health checks. My GitHub repo has all the Docker and Docker Compose files, the application configuration, and so on. I’ve added a MySql database and a Redis cache to my Docker Compose application to make this tutorial easier to follow. I’d recommend pulling the repo and having a look so you can see how this is all done for realsies.

MySql Health Check

Let’s begin by adding a health check for a MySql database. Start by adding the following package to your project.

Install-Package AspNetCore.HealthChecks.MySql

In order to test the connection to our database, we need to add a connection string to our appsettings.json configuration file. You’ll see how this configuration file is loaded in the startup.cs file of my example repo. Let’s add the following line to the bottom of our appsettings.json file

“Data”: {
 “ConnectionStrings”: {
 “MySql”: “Server=mysqldb;Database=db;Uid=user;Pwd=password;”
 }

An aside: Where is that database name coming from?
For those of you unfamiliar with Docker and Docker Compose, in a composed containerised application, each child container can automatically reference its siblings by their name defined in the docker-compose file. That is brilliantly convenient, as it means I do not need to think about this kind of naming, as the docker-compose engine makes this transparent to me.

In your startup.cs, let’s add now add the following code to your ConfigureServices method, as so:

services.AddHealthChecks()
 .AddMySql(Configuration[“Data:ConnectionStrings:MySql”]);

This is telling my MySql Health Check to call the database connection string and see if it can access this database.

Fire up your application by right-clicking on the docker-compose project, and choosing “Set as Default Project”, then clicking “Docker Compose” start button at the top.

(note: the first time you run this, it might take a moment, as Docker compose must pull and initialise the MySql Docker container to get it all up and running.)

Once my app is running, navigate to http://localhost:54411/health

And you should get this back.

{
  "status": "Healthy",
  "results": {
    "mysql": {
      "status": "Healthy",
      "description": null,
      "data": {}
    }
  }
}

That’s nice! Our Health Check called out to the database, found it could connect, and thus returned healthy.

Redis Health Check

Let’s now add a health check for our Redis cache. Add the following package to your project.

Install-Package AspNetCore.HealthChecks.Redis

In order to test the connection to our database, we need to add a connection string to our appsettings.json configuration file as we did before.

“Data”: {
 “ConnectionStrings”: {
 “MySql”: “Server=mysqldb;Database=db;Uid=user;Pwd=password;”,
 “Redis”: “redis:6379”
 }
 }

And let’s add our health check to our startup.cs file, as so

services.AddHealthChecks()
 .AddMySql(connectionString: Configuration[“Data:ConnectionStrings:MySql”])
 .AddRedis(redisConnectionString: Configuration[“Data:ConnectionStrings:Redis”],
 failureStatus: HealthStatus.Degraded);

See how we made the status degraded simply by changing the failureStatus type? Now, let’s fire up our app again, and because everything is running, we should see this on our health check endpoint

{
  "status": "Healthy",
  "results": {
    "mysql": {
      "status": "Healthy",
      "description": null,
      "data": {}
    },
    "redis": {
      "status": "Healthy",
      "description": null,
      "data": {}
    }
  }
}

Our API is healthy because it can access both MySql and Redis.

Okay, so let’s simulate a failure, and shut down our redis instance from the command line on Docker. You can stop the redis instance by first finding it from a docker-enabled command prompt.

docker ps | grep redis

This will give us a listing like this

8a43e10dc0a3 redis “docker-entrypoint.s???” x minutes ago Up y seconds 6379/tcp

You’ll see a unique hash key like I’ve highlighted (yours will be different from mine). Grab that hash key at the beginning, and run the following command

docker stop 8a43e10dc0a3

Let’s run our health check again. This time, we will see our health check is degraded, because our microservice cannot reach redis.

{
  "status": "Degraded",
  "results": {
    "mysql": {
      "status": "Healthy",
      "description": null,
      "data": {}
    },
    "redis": {
      "status": "Degraded",
      "description": null,
      "data": {}
    }
  }
}

PS: If you want to start that redis cache back up, just run the docker start command from the command prompt using the same hash you used to stop it. The container is stopped, not deleted.

docker start 8a43e10dc0a3

So now we have a health check for MySql and for Redis. Let’s implement our own health check.

Writing a custom health check all your own

In the last post, I showed you how to create a phony baloney health check, but which did literally nothing. Let’s add some meat and potatoes to that same command. We’ll add a file like last time called ApiHealthCheck.cs. Here’s the code from my last post but modified to actually do something. It’s going to look for a content file in the app running directory called “MyApiLicenseFile.txt” (A stupid and contrived example, I know ,but it serves to give us an idea of what can be done). If the file exists, the API is healthy. Otherwise, it’ll barf.

public class ApiHealthCheck : IHealthCheck
 {
 public Task<HealthCheckResult> CheckHealthAsync(
 HealthCheckContext context,
 CancellationToken cancellationToken = new CancellationToken())
 {
 //we’re looking to see if a file called “MyApiLicenseFile.txt exists in the
 //root directory of our API. If not, then this is unhealthy.
 //If the file does exist, then all good and healthy.
 var basePath = AppDomain.CurrentDomain.BaseDirectory;
 var isHealthy = File.Exists($”{basePath}MyApiLicenseFile.txt”);
 if(isHealthy)
 {
 return Task.FromResult(HealthCheckResult.Healthy(“I am one healthy microservice API”));
 } return Task.FromResult(HealthCheckResult.Unhealthy(“I am the sad, unhealthy microservice API. Cannot find my license file ‘MyApiLicenseFile.txt’”));
 }
 }

Now add a reference to this ApiHealthCheck to our startup.cs as so

services.AddHealthChecks()
 .AddMySql(connectionString: Configuration[“Data:ConnectionStrings:MySql”])
 .AddRedis(redisConnectionString: Configuration[“Data:ConnectionStrings:Redis”],
 failureStatus: HealthStatus.Degraded)
 .AddCheck<ApiHealthCheck>(“api”);

And fire up the API using the Docker-Compose start as before. Since the file the API is looking for doesn’t exist, we’ll get this back on our health check endpoint

{
  "status": "Unhealthy",
  "results": {
    "mysql": {
      "status": "Healthy",
      "description": null,
      "data": {}
    },
    "redis": {
      "status": "Healthy",
      "description": null,
      "data": {}
    },
    "api": {
      "status": "Unhealthy",
      "description": "I am the sad, unhealthy microservice API.  Cannot find my license file 'MyApiLicenseFile.txt'",
      "data": {}
    }
  }
}

Just as we suspected, yeah? Okay, so let’s stop the API, and add a file with that name. It can be empty, but you must open the properties for the file, and set its Build action as “Content”, and Copy to Output Directory to “Copy Always”

Now fire up the API. It will find the MyApiLicenseFile.txt as expected, and all will be well.

{
  "status": "Healthy",
  "results": {
    "mysql": {
      "status": "Healthy",
      "description": null,
      "data": {}
    },
    "redis": {
      "status": "Healthy",
      "description": null,
      "data": {}
    },
    "api": {
      "status": "Healthy",
      "description": "I am one healthy microservice API",
      "data": {}
    }
  }
}

Conclusion

As you can see, building health checks using prebuilt health checks, as well as rolling your own is really a breeze. You can ensure your applications are healthy with very little effort, and guarantee that when things do go wrong, you can be informed of it in a neat and reliable manner.

Hope this helps