Infection — Mutation Testing Framework

Basics of Mutation Testing

  • Negating conditionals;
  • Changing return values;
  • Changing method’s visibility;
  • and so on.
  1. The line of mutated code is not covered by tests;
  2. Tests are not very useful for this line of code.
./infection.phar --threads=4

How is MSI calculated?

Metrics:     
Mutation Score Indicator (MSI): 47%
Mutation Code Coverage: 67%
Covered Code MSI: 70%

Mutation Score Indicator (MSI)

TotalDefeatedMutants = KilledCount + TimedOutCount + ErrorCount;MSI = (TotalDefeatedMutants / TotalMutantsCount) * 100;

Mutation Code Coverage

TotalCoveredByTestsMutants = TotalMutantsCount - NotCoveredByTestsCount;  CoveredRate = (TotalCoveredByTestsMutants / TotalMutantsCount) * 100;

Covered Code Mutation Score Indicator

TotalCoveredByTestsMutants = TotalMutantsCount - NotCoveredByTestsCount;TotalDefeatedMutants = KilledCount + TimedOutCount + ErrorCount;CoveredCodeMSI = (TotalDefeatedMutants / TotalCoveredByTestsMutants) * 100;

When you write tests for the code with intervals, you must test boundary values.

./infection.php --threads=4 --show-mutations --mutators=PublicVisibility,ProtectedVisibility

Infection PHP

./infection.phar self-update
./infection.phar --mutators=PublicVisibility,Plus,Decrement
./infection.phar --min-msi=80 --min-covered-msi=95

Using Infection with Travis CI

before_script:
- wget https://github.com/infection/infection/releases/download/0.6.0/infection.phar
- wget https://github.com/infection/infection/releases/download/0.6.0/infection.phar.pubkey
- chmod +x infection.phar script:
- ./infection.phar --min-covered-msi=90 --threads=4

How to use Mutation Testing?

Daily basis usage for developer

  • You write a new class, e.g. that UserFilterAge;
  • This class is already covered by tests;
  • To check the efficiency of these tests, you run MT just for this file.
./infection.phar --threads=4 --filter=UserFilterAge.php --show-mutations

Daily basis usage for project

Sometimes it is not possible to have 100% MSI

Abstract Syntax Tree

  • Split given source into PHP tokens (function token_get_all()) and store them in an array;
  • Loop through these tokens and decide whether particular token should be replaced according to one of the mutation operator or not;
  • Reconstitute the new source code (mutant) from the updated array of tokens.
T_OPEN_TAG ('<?php ')
T_BOOLEAN_AND ('&&')
T_INC ('++')
T_WHITESPACE (' ')
...
  • Are we in the function body? (we don’t want to replace T_OPEN_TAG ('<?php '), right?);
  • Will the mutated code be valid after mutation is done? For example, arrays union ['a'] + ['b'] is a valid code, but arrays subtraction ['a'] — ['b'] is a Fatal Error. This mutation should be skipped by MF.
  • Much easier to support code;
  • Much easier to write new Mutators;
  • Much easier to handle false-positives and different edge cases, e.g. deciding when mutation should be done or should not in a difficult situation.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store