In my last blog post, I discussed the advantages of running acceptance tests at a level where it is still possible to isolate the code under test from third-party dependencies. Here I’m going to detail (using real-life examples) how Varealis has designed its .Net Core applications to run acceptance tests against one of the microservices that helps power RadSpider.
ASP.Net Core Acceptance Tests
ASP.Net Core offers an in-memory TestServer
, made available in the Microsoft.AspNetCore.Mvc.Testing
nuget package, to allow for integration/acceptance tests to be run without deploying the application to a test environment. A WebApplicationFactory
is also provided to streamline the bootstrapping the TestServer
and generate a HttpClient
that can be used to interact with the web application. For more information about this Microsoft has provided detailed documentation which can be found here.
Generating a Client
By initialising a WebApplicationFactory
using the same Startup
class that is used to bootstrap the web application, we can generate a basic HttpClient
that can interact with the web application.
var pipelineId = Guid.NewGuid();var httpClient = new WebApplicationFactory<Startup>().CreateClient();
HttpOperationResponse httpResponse = await httpClient
.GetAsync($"/Pipeline/{pipelineId}");
While this client gives us everything we need to be able to integrate with a web application, since my web applications also automatically generate OpenAPI swagger files using Swashbuckle.AspNetCore
I like to use this file to help generate a HttpClient
. Not only does this greatly simplify the interactions with the service under test, but it also makes it possible to verify that our documentation effectively describes how to interact with that service. We are using the tools that we will later provide our customers in order to test our application, even at an acceptance test level.
To do this, I used ‘ autorest’ to automatically generate a client that can be used to interact with the web application. The HttpClient
generated by the WebApplicationFactory
can then be used to initialise the 'autorest' client which provides a friendly interface which can be used to integrate with the service.
var httpClient = new WebApplicationFactory<Startup>().CreateClient();// This PipelineClient was automatically generated by autorest
var pipelineClient = new PipelineClient(
new ClientCredentials(),
httpClient,
true);HttpOperationResponse<PipelineDetails> pipeline = await pipelineClient
.GetPipelineDetailsWithHttpMessagesAsync(Guid.NewGuid());
* Note that automatically generated extension methods can also be used when we don’t care to know about the HttpOperationResponse
and only the response body. These look something like this...
PipelineDetails pipeline = await pipelineClient
.GetPipelineDetails(Guid.NewGuid());
Mocking Dependencies
So far, because we are using the application’s original Startup
class, we are also relying on all of the original application's dependencies. For these tests to work in instances where we don't have access to third-party dependencies like databases and/or service buses we need to replace these dependencies with in-memory alternatives or mocks. To achieve this we have to make some minor changes to the Startup
class to allow selected dependencies to be overridden. There are many ways of doing this, but I did this by introducing a base class. The StartupBase
class is then responsible for configuring all of the fixed internal dependencies while the now elevated Startup
is now responsible for adding/configuring the dependencies that we want to override.
public class Startup : StartupBase
{
protected override IServiceCollection ConfigurePipelineRepository(IServiceCollection services)
{
var connectionString = this.Configuration
.GetSection("Persistence")
.GetValue<string>("PipelineRepository"); var mongoUrl = new MongoUrl(connectionString);
var mongoSettings = MongoClientSettings
.FromUrl(mongoUrl); services.AddSingleton<IMongoClient>(
new MongoClient(mongoSettings)); services.AddSingleton<IPipelineRepository, PipelineRepository>(); return services;
}
}public abstract class StartupBase
{
protected IConfiguration Configuration { get; } // A lot of this method has been removed to maintain brevity
public void ConfigureServices(IServiceCollection services)
{
services.AddMvc();
services.AddSwaggerGen(); this.ConfigurePipelineRepository(services);
}
protected abstract IServiceCollection ConfigurePipelineRepository(IServiceCollection services);
}
By doing this any class that extends the StartupBase
is now able to configure mocked dependencies in place of third-party dependencies that we don't want to rely on at this stage. We do this by creating a special MockStartup
class which injects mocks into the IServiceCollection
in place of the real implementations.
using NSubstitute;public class MockStartup : StartupBase
{
protected override IServiceCollection ConfigurePipelineRepository(IServiceCollection services)
{
services.AddSingleton(
_ => Substitute.For<IPipelineService>()); return services;
}
}
This approach is not perfect, and there are two important things to note here. Firstly, whenever dependencies are replaced with mocks you are limiting the scope that the tests can cover. Ideally, you should look to replace dependencies with in-memory implementations of dependencies rather than mocks. For example, if you are using Entity Framework Core then you can use the in-memory database provider in place of your usual provider. This allows you to cover much of the database interactions at this level than a mock would allow since you are still interacting with a ‘real’ database. In my case, I am forced to mock out my IPipelineRepository
because it wraps a mongo database which doesn't have (at the time of writing) an equivalent to this. Unfortunately this means that a lot of database interactions that I would love to test at this level (like the QueryBuilder
s) can not be tested at this level of the testing triangle.
Secondly, we need to be careful to reduce the amount of work that is done in the override methods, so steer clear of broad methods such as protected abstract IServiceCollection ConfigureDependencies(IServiceCollection services)
instead favouring many smaller methods for each dependency that you are trying to replace. Because the override methods implemented in the Startup
class are not covered by the tests it can be easy to forget/misconfigure dependencies in the real methods and be lulled into a false sense of security that these tests can provide.
All of this goes to say that, while these tests are extremely valuable, they can not replace full end-to-end integration tests. Hopefully, you can drastically reduce the number of more expensive integration tests that you need to run, especially since these tests allow you to easily manufacture scenarios that may not otherwise we easy to replicate with real dependencies.
Other Applications
In cases where you can’t host your application in an ASP.Net in-memory test server, you have to get a little more creative. In my case, I also have ‘ MassTransit’ consumers which also use Startup
style classes to initialise the IBusControl
. However, since these consumers are hosted in Service Fabric ICommunicationListener
s, I am not able to load them into a TestServer
like I did before. Fortunately, all we really have to do here is make it possible to override MassTransit's transportation layer on top of the third-party dependencies.
public class Startup : StartupBase
{
public Startup()
{
var connectionString = this.Configuration
.GetSection("Persistence")
.GetValue<string>("SagaDatabase");var mongoUrl = new MongoUrl(connectionString);
var mongoSettings = MongoClientSettings.FromUrl(mongoUrl);this.sagaDatabase = new MongoClient(mongoSettings)
.GetDatabase("Pipeline");
}protected override ISagaRepository<PipelineSaga> PipelineSagaDatabase =>
new MongoDbSagaRepository<PipelineSaga>(
this.sagaDatabase,
new MongoDbSagaConsumeContextFactory(),
nameof(PipelineSaga));protected override IBusControl
ConfigureMassTransit(IServiceCollection services)
{
var messageBusSettings = this.Configuration.GetSection("ServiceBus");return Bus.Factory.CreateUsingAzureServiceBus(cfg =>
{
var host = cfg.Host(
messageBusSettings.GetValue<string>("ConnectionString"),
config => { });cfg.ReceiveEndpoint(
host,
this.RecieveEndpointQueue,
ec => this.ConfigureRecieveEndpoint(
ec,
services.BuildServiceProvider()));
});
}protected override IServiceCollection
ConfigurePipelineRepository(IServiceCollection services)
{
var connectionString = this.Configuration
.GetSection("Persistence")
.GetValue<string>("PipelineRepository");
var mongoUrl = new MongoUrl(connectionString);
var mongoSettings = MongoClientSettings.FromUrl(mongoUrl);services.AddSingleton<IMongoClient>(new MongoClient(mongoSettings));
services.AddSingleton<IPipelineRepository, PipelineRepository>();return services;
}
}public abstract class StartupBase
{
protected IConfiguration Configuration { get; }protected string RecieveEndpointQueue =>
"Varealis.RadSpider.Services.Pipeline.Consumer";protected abstract ISagaRepository<PipelineSaga> PipelineSagaDatabase { get; }// A lot of this method has been removed to maintain brevity
public void ConfigureServices(IServiceCollection services)
{
this.ConfigurePipelineRepository(services);
services.AddSingleton<PipelineConsumer>();services.AddSingleton(_ => this.ConfigureMassTransit(services));
services.AddSingleton<IPublishEndpoint>(provider =>
provider.GetRequiredService<IBusControl>());
}protected void ConfigureRecieveEndpoint(
IReceiveEndpointConfigurator recieveConfig,
IServiceProvider serviceProvider)
{
recieveConfig.Consumer(() => serviceProvider.GetRequiredService<PipelineConsumer>());
recieveConfig.Saga(this.PipelineSagaDatabase);
}protected abstract IServiceCollection
ConfigurePipelineRepository(IServiceCollection services);protected abstract IBusControl
ConfigureMassTransit(IServiceCollection services);
}
From this point, all I have to do is create a special MockStartup
class that extends BaseStartup
and implements all of the abstract methods. However, in place of CreateUsingAzureServiceBus
I'm going to use CreateUsingInMemory
. This tells MassTransit to use MassTransit's In-Memory transportation layer, which means I'm able to run the consumer without having a dependency on a service bus like RabbitMQ and Azure Service Bus.
public class MockStartup : StartupBase
{
private readonly InMemorySagaRepository<PipelineSaga> pipelineSagaDatabase;public MockStartup(InMemorySagaRepository<PipelineSaga> pipelineSagaDatabase)
{
this.pipelineSagaDatabase = pipelineSagaDatabase;
}protected override ISagaRepository<PipelineSaga> PipelineSagaDatabase =>
this.pipelineSagaDatabase;protected override IBusControl
ConfigureMassTransit(IServiceCollection services)
{
return Bus.Factory.CreateUsingInMemory(cfg =>
{
cfg.ReceiveEndpoint(
this.RecieveEndpointQueue,
ec => this.ConfigureRecieveEndpoint(
ec,
services.BuildServiceProvider()));
});
}protected override IServiceCollection
ConfigurePipelineRepository(IServiceCollection services)
{
services.AddSingleton(_ => Substitute.For<IPipelineService>());
return services;
}
}
Happy Testing
I hope that these two examples can help you experiment with this approach to testing since I’ve found it invaluable when it comes to testing microservices. For one of my microservices, at the time of writing, I have 388 unit tests which can be run in 2.61 seconds and then a further 92 acceptance tests that can be run in 6.39 seconds. These acceptance tests have enabled me to catch a number of bugs with how the code was integrated, bugs that are very difficult to find with unit tests alone. Not bad considering all 480 tests can be run in 7 seconds when running in parallel, and that these tests can be run anywhere.
Originally published at https://www.varealis.com.