Avoid Unintended Coverage in PHPUnit
When it comes to writing tests for your PHP code base, a good measure of the quality of your tests is the PHPUnit coverage report. While this metric is essential to help you identify which portions of your code need more testing, it can also be misleading through what I’d like to call “unintended coverage”.
Unintended Coverage
Imagine that you have written the following piece of code, you have an Order Controller which also applies a potential discount:
class DiscountCalculator
{
const REGULAR_DISCOUNT = 0.1;
const SPECIAL_DISCOUNT = 0.2;
public static function calculate($amount, $type)
{
if ($type == 'regular') {
return $amount - ($amount * self::REGULAR_DISCOUNT);
} elseif ($type == 'special') {
return $amount - ($amount * self::SPECIAL_DISCOUNT);
}
}
}
class OrderController
{
public function post($orderData)
{
$amount = $orderData['amount'];
$type = $orderData['type'];
// Other logic (validation, calls to shipping APIs, etc.)
$discountedAmount = DiscountCalculator::calculate($amount, $type);
$order = new Order($amount, $discountedAmount);
$order->save();
}
}In the above pseudo-code and while writing tests, we can safely assume that a test for OrderController::post would exist even if one for DiscountCalculator does not. As the given post request is essentially the final product of our pseudo-API.
class OrderControllerTest
{
public function testOrderIsSavedForRegularCustomer()
{
$orderData = [
'amount' => 100,
'type' => 'regular'
// ...
];
$orderController = new OrderController();
$orderController->post($orderData);
// Assert that the order was saved
}
public function testOrderIsSavedForPremiumCustomers()
{
// ...
}
}Yet if you actually test this code with PHPUnit, you will not only get a coverage result for OrderController, you’d see that entire portion of code covered. This may seem like the intended behavior of how the coverage is calculated, after all, the coverage number and report indicate which portion of your code was executed through the test and it may help identify completely untested code, but the problem with unintended coverage remains: you never wrote tests with intention to check for the output and behavior of another part of your system. (e.g. DiscountCalculator) a miscalculation for example, either needs to be caught in the broader tests (asserting that the price was calculated correctly in OrderControllerTest) or it may not be caught at all depending on what you’re testing in the broader sense. Worst of all? it will show up as green and tested in your reports!
Unintended coverage is seeing portions of code as “covered” where you never wrote tests intentionally to cover those portions.
Finding Unintended Coverage Through Annotations
Now that we have covered the definition of what counts as unintended coverage, in order to have code that we have actually, mindfully tested, we’d like to identify and test these mislabeled portions of code. To find these portions, we will need to specify what we are trying to cover in each test that we write. Luckily, PHPUnit has us covered with some PHPDoc annotations and PHP attributes to help us specify our intended pieces of code for each test.
Being Precise in Unit Tests Through Coverage Tags
The first, simplest type of annotation is @covers. This tag can mark the class you intend to test, making PHPUnit exclude it from the coverage of other portions of code, in our example, adding covers would look like so:
/**
* @covers \OrderController
*/
class OrderControllerTest
{
public function testOrderIsSaved()
{
$orderData = [
'amount' => 100,
'type' => 'regular'
// ...
];
$orderController = new OrderController();
$orderController->post($orderData);
// Assert that the order was saved
}
}if we once again run the PHPUnit coverage tests and take a look at the code, we will find that our original Discount Calculator class will now correctly be marked as uncovered.
Using @coversDefaultClass and @covers if further precision is needed
Treat with Caution! PHPUnit documentation does not recommend marking tests as covering only a specific method of a class, and this seems to be unsupported in the new attributes as well. It’s understandable considering the below example does not provide a strong use case for this behavior either.
Now let’s say that we would like to have a function in the OrderController class to actually view a submitted order, in this sample scenario, we would need to also call the post method of the OrderController to save an order, like so:
/**
* @covers \OrderController
*/
class OrderControllerTest
{
public function testOrderIsSaved()
{
$orderData = [
'amount' => 100,
'type' => 'regular'
// ...
];
$orderController = new OrderController();
$orderController->post($orderData);
// Assert that the order was saved
}
public function testGetOrder()
{
$orderData = [
'amount' => 100,
'type' => 'regular'
// ...
];
$orderController = new OrderController();
$orderController->post($orderData);
$order = $orderController->get(1);
// Assert that the order was retrieved
}
}In this scenario, testGetOrder would still generate unwanted coverage because we are calling the post method again, possibly with some variance that isn’t intentionally tested in testOrderIsSaved, we can get more precise in which method of a function we intend to test by using @coversDefaultClass in combination with @covers:
/**
* @coversDefaultClass \OrderController
*/
class OrderControllerTest
{
/**
* @covers ::post
*/
public function testOrderIsSaved()
{
$orderData = [
'amount' => 100,
'type' => 'regular'
// ...
];
$orderController = new OrderController();
$orderController->post($orderData);
// Assert that the order was saved
}
/**
* @covers ::get
*/
public function testGetOrder()
{
$orderData = [
'amount' => 100,
'type' => 'regular'
// ...
];
$orderController = new OrderController();
$orderController->post($orderData);
$order = $orderController->get(1);
// Assert that the order was retrieved
}
}@coversDefaultClass will tell PHPUnit that generally, the tests in this case are intended for OrderController, while @covers tags will tell PHPUnit precisely which method of the class should be covered by the code. While you could also use the full annotation for each @ covers can also be used as below:
/**
* @covers \OrderController::get
*/
public function testGetOrder()
{
$orderData = [
'amount' => 100,
'type' => 'regular'
// ...
];
$orderController = new OrderController();
$orderController->post($orderData);
$order = $orderController->get(1);
// Assert that the order was retrieved
}Tests With No Coverage
There are scenarios where you’d like your test to not contribute to the coverage report at all. Imagine in our OrderController example that we’d like to create an Order, and perform assertions to check that the shipping API was called but we do not intend for this test to contribute to the coverage, in which case, PHPUnit supports a tag called @coversNothing:
/**
* @coversNothing
*/
public function testOrderIsSavedWithShippingAPICalled()
{
$orderData = [
'amount' => 100,
'type' => 'regular'
// ...
];
$orderController = new OrderController();
$orderController->post($orderData);
// Perform assertions on the shipping API
}This test will be excluded from the coverage calculations regardless of which parts of the code it executes.
Using New PHP 8 Attributes to Mark Coverage
All the code above is intended to work with both PHP 7.4 and 8.0+, yet if you are working on a newer PHP code base, you may use attributes instead of annotations:
use PHPUnit\Framework\Attributes\CoversClass;
use PHPUnit\Framework\Attributes\CoversNothing;
class OrderControllerTest
{
#[CoversClass(OrderController::class)]
public function testOrderIsSaved()
{
$orderData = [
'amount' => 100,
'type' => 'regular'
// ...
];
$orderController = new OrderController();
$orderController->post($orderData);
// Assert that the order was saved
}
#[CoversNothing]
public function testOrderIsSavedWithShippingAPICalled()
{
$orderData = [
'amount' => 100,
'type' => 'regular'
// ...
];
$orderController = new OrderController();
$orderController->post($orderData);
// Perform assertions on the shipping API
}
#[CoversClass(OrderController::class)]
public function testGetOrder()
{
$orderData = [
'amount' => 100,
'type' => 'regular'
// ...
];
$orderController = new OrderController();
$orderController->post($orderData);
$order = $orderController->get(1);
// Assert that the order was retrieved
}
}Being Strict For New or Mature Projects
So far, we have covered what unintentional coverage is and how it can be avoided, but it is inevitable that without proper checks and with large enough teams, it becomes one more reviewing task to ensure that no tests lack the annotations for what they cover. Whether you are working on a brand-new PHP project or you’ve successfully refactored an existing project with coverage tags, you can ensure that all new tests have one of the above coverage annotations using the new configuration option of “requireCoverageMetadata” which when set true in phpunit.xml or as “strict-coverage” in command line attributes, will ensure that all tests being run by PHPUnit have coverage annotations or attributes present for them. More information about this can be viewed on PHPUnit documentation.
Final Thoughts
With the increasing reliance of software developers on methods such as TDD to write more reliable code, we can utilize the functionalities provided by PHPUnit to increase confidence in your test coverage significantly. If you’d like to learn more about the subject, feel free to take a look at the following resources from the PHPUnit documentation as well:
- Unintentionally Covered Code
- @covers, @coversDefaultClass and @coversNothing
- New Code Coverage Attributes
- Code Coverage
Do you have any questions or would you like to see another topic covered? Or perhaps this post needs to be updated? Please leave a comment below.
