Applying TDD to reduce defects in complex domains
Introduction
Will I benefit from this article?
Before you dive into the article I wanted to give a short summary of what we will cover, since the problem space is very specific. You will benefit from it if:
- You use Domain Driven Design
- Have fairly complex evolving domains
- Are familiar with Test Driven Development
- Are familiar with Behavior Driven Development (in the article I mention specifically ITLibrum BDD Toolkit, more on that: here)
- Are familiar with automatic test data generation (in our team we use AutoFixture library for that. link)
Business Background
Some time ago our team at EcoVadis took over an important part of the domain of our core application. Team also got involved in a large project of optimization and automation of the evaluation process — a part of our core business.
The project aims to introduce automation of some parts of the scoring process, allowing the company to increase the capabilities and analyst throughput. Our team was responsible for automation of assignment of some specific parameters to the evaluation.
Code Landscape
As a part of that, we’ve created an automation engine for the evaluation attributes calculations, with a couple of key principles and assumptions kept in mind:
- Business rules may evolve in time.
- In the future we will want to create a “builder” for end users, allowing them to define their own rules for calculations.
- We wanted to make sure that at any given point in time we will be able to easily plug in and create additional criteria.
- When we took over the domain part, we inherited an “old” automation solution which covered some part of our scope. We had to make sure that we will be able to migrate that part into our new automation engine.
With all of the following assumptions in mind, we’ve came up with solution looking as following:
There is a lot going on here, so let me give you a walkthrough.
- Rule class is the base of our solution, it contains references to the context of the rule as well as a set of RuleCriteria that need to be met in order for the rule to be applicable.
- RuleContext defines set of contextual data shared across all of the criteria
- RuleCriterion defines single criteria of evaluation (ex. We would like to apply certain attributes only to suppliers with certain sizes)
- RuleCriterionEvaluator and extending classes are responsible for verification whether certain criterion is applicable
- EvaluationObjects are responsible to hold actual value which we check against
- RuleEvaluator is responsible for checking applicability of rule as a whole
To give you better understanding, considering the information above we can formulate example business rule for automation solution.
“If the company has size X and was granted score Y in some area, attribute “ABC123” should be assigned.”
Our Challenge
As you can see we’ve ended up with a fairly complex solution with a lot of generics and abstraction on top of it, so naturally during the implementation things we’ve started to stumble upon an increasing number of bugs and defects in our code.
Naturally we wrote both — unit and integration tests as well as did manual testing of the newly implemented automation rules. Our first pain point was right at this place. Even though we tested our code and had good code coverage, we were writing our tests post factum, allowing some of the defects to slip under the radar. As we started to subconsciously write tests which were simple and just “good enough” to pass.
In our code we have also ended up with a couple of “configurable criteria” classes. Those were changing and evolving over time (for example new properties had to be added) — because of new business requirements.
We realized that creating tests afterwards (after extending the criteria class with new business functionality) was no longer a viable option for us. Any change in those classes had an impact on functionalities which were already up and running on production. If we didn’t analyze how functionality extension would impact existing tests we could easily receive “false positives” in test results and promote broken code to production.
Reducing the defects
TDD as a potential solution
Afterwards we’ve realized that we could mitigate a lot of the issues by better anticipating the changes. Since in EcoVadis we have quite extensive tests framework TDD was to us an obvious way to address the challenge.
We’ve decided to apply TDD mainly to the unit tests of specific criteria, since criteria changes were in most cases the origin of the issue.
In most of the TDD articles that you can find on the internet you can find simple approach to TDD:
However, as mentioned before we had an additional challenge, there was already an existing code base. That meant we’ve had to take an additional step before writing new tests.
We had to anticipate how existing tests would react to our changes, in order to make sure that we do not introduce new issues to the code.
It is important to write down those cases and expected behavior, reducing the possibility of missing something (even pen & paper will do the job!).
After you do that you can continue with the standard TDD approach and adjust the existing test methods where it is necessary.
Example
As an illustration of our process let’s examine following simplified Criterion implementation:
public class SizeRuleCriterion : RuleCriterion<SizeEvaluationObject>
{
public IReadOnlyCollection<CompanySize> Sizes { get; }
public SizeRuleCriterion(IReadOnlyCollection<CompanySize> sizes)
{
if (sizes == null || !sizes.Any())
{
new MissingCompanySizesForSizeCriterionException();
}
Sizes = sizes;
}
public override bool Check(SizeEvaluationObject evaluationObject, RuleContext ruleContext)
{
return Sizes.Contains(evaluationObject.Size);
}
}And our corresponding evaluation object:
public class SizeEvaluationObject : IEvaluationObject
{
public CompanySize Size { get; set; }
}Our criterion is performing validation based on the presence of specific CompanySize in the evaluation object. We also have already existing test methods for the criterion:
[Fact]
public void WhenCompanySizeIsNotInCriteriaList_CriterionNotMet() =>
BddScenario.Using<Context>()
.Given(_ => _.SomeEvaluation())
.And(_ => _.HasSize(CompanySize.S))
.And(_ => _.SizeCriterion(CompanySize.M, CompanySize.L))
.When(_ => _.EvaluatingRuleCriterion())
.Then(_ => _.CriterionNotMet())
.Test();
[Fact]
public void WhenCompanySizeIsInCriteriaList_CriterionMet() =>
BddScenario.Using<Context>()
.Given(_ => _.SomeEvaluation())
.And(_ => _.HasSize(CompanySize.M))
.And(_ => _.SizeCriterion(CompanySize.M, CompanySize.L))
.When(_ => _.EvaluatingRuleCriterion())
.Then(_ => _.CriterionMet())
.Test();Now let’s suppose that new requirements come in, our business requested that validation of our criterion in some cases should be also based on combination of company size and origin country. This means that our “to be” structure would look as follows:
public class SizeRuleCriterion : RuleCriterion<SizeEvaluationObject>
{
public IReadOnlyCollection<CompanySize> Sizes { get; }
public Country OriginCountry { get; }
public SizeRuleCriterion(IReadOnlyCollection<CompanySize> sizes, Country country)
{
if (sizes == null || !sizes.Any())
{
new MissingCompanySizesForSizeCriterionException();
}
OriginCountry = country;
Sizes = sizes;
}
public override bool Check(SizeEvaluationObject evaluationObject, RuleContext ruleContext)
{
// New logic for evaluating criterion
}
}Following the TDD, we should do following things:
- Create test methods to test the functionality when “OriginCountry” property is present.
- Modify our BDD context methods so that they allow setting our new property, which we plan to add (ex. SizeRuleCriterion method)
- Run the tests
- Modify our SizeRuleCriterion class and add the new property
- Change logic of Check method
- Run the tests
- Refactor (if necessary)
- Run tests again
But, if we just follow a standard pattern, we must perform one more step in between, to make sure that we don’t introduce new defects.
After creating the test methods for OnlyValid functionality, we should carefully assess existing test methods. — So, we need to ask ourselves a questions:
- Considering requirement changes, should my existing methods still return the same results?
- Should the tests that are currently green, still remain green after we implement the changes?
- What kind of validations do we need to add to existing tests?
This is necessary to eliminate false positive results, which may lead us to deployment of faulty code to the production environment.
Conclusion
The exercise of TDD application to our domain unit tests was very valuable for our team.
Applying TDD in a correct way adjusted to the project reality is certainly difficult, yet successful for us.
Our key takeaways from the whole exercise of introduction of TDD in our code were:
- It is essential to define expected outcome for existing test methods before implementing new test methods and modifying the business logic
- When using AutoFixture it is worth to use “Run Unit Test until fail” option in the ReSharper Test Runner, to minimize risk of creating non-deterministic tests.
- In Domain Driven Design code landscape, we often work with bigger classes containing numerous properties. Since the domain objects are built through constructors (and contain only get properties) it is tough to mock them using libraries such as AutoFixture and maintain the test methods small and concise. Given that, Builder classes should be introduced, allowing to increase readability of the test classes.
- Using BDD framework for us had it’s pros and cons:
Pros:
— Test methods (excluding context class) were very easy to read and understandable at a first glance
Cons:
— BDD framework for unit tests in our case resulted in large context classes with a lot of method definitions which became harder and harder to navigate this could be avoided by either:
— — Defining separate classes for each flow path
— — As silly as it sounds — resign from the BDD framework and come back to the “classic” way to write unit tests
