Fault Seeding — Why it’s a good idea to insert bugs in your software (Part 1)

This is the first blog post in a series about Fault Seeding. Last year I wrote my master’s thesis about this topic and so I want to share the results of my paper with the developer and tester community. Part 1 contains a general description of Fault Seeding. All further parts will give you a deeper understanding of Fault Seeding.


Fault Seeding, also known as Error Seeding, Defect Seeding or Bebugging is a method in the software development lifecycle to evaluate the Fault Detection Rate of software tests and the number of remaining faults in the software. These two metrics provide a deeper understanding about the test quality and software quality.

Fault Detection Rate

This metric is a measure of the testing effectiveness and is one important aspect of test quality. It answers the following question: How many faults (in %) are found by my software tests?

Number of Remaining Faults

This software metric provides information on the number of remaining faults after the tests. It is one aspect of software quality and answers the question: How many faults after test activities remain in my software?


The idea behind Fault Seeding is to simply insert artificial faults into the software and after testing count the number of artificial and real faults discovered by the software tests. The complete process of Fault Seeding consists of the following five steps:

  • Fault Modeling
  • Fault Injection
  • Test Execution
  • Calculation
  • Interpretation

Fault Modeling

The first step is to model some artificial faults. You can e.g. replace arithmetic, relational or logical operators, remove function calls or change the datatype of variables. There are many types of faults, be creative.

To make it absolutely clear, here is an example. Image 1 shows the original source code. If variable a is greater than 5, function func() is executed.

Image 1: Original source code

Now we model a new artificial fault by manipulating the original source code which is seen in image 2. If variable a is equal or greater 5, function func() is executed.

Image 2: Manipulated source code (contain an artificial fault)

Fault Injection

These artificial faults are then seeded into the software. The number of seeded faults is self-determined and depends on the size of the software. For example, if your software has 1,000 lines of code five artificial faults might be insufficient. An exact rule to determine the number of artificial faults does not exist, sorry folks.

The injection of artificial faults into the master branch should be avoided. It is recommended to create a new branch and do the fault seeding activities in this branch.

Test Execution

Now your manual or automated tests are executed. The aim of the tests is to discover as much faults as possible.


Your tests hopefully discover a proper number of real and artificial faults. Now, the number of discovered seeded faults, the number of seeded faults and the number of discovered real faults are known. In the following mathematical equations you can see the calculation of the Fault Detection Rate and the number of the remaining faults. The following mathematical equation represents the basic relationship between seeded and real faults which the Fault Seeding method is based on.

The other mathematical equations shows the calculation of the Fault Detection Rate and the number of remaining faults which are derived from the first equation.


The main objective of Fault Seeding is to evaluate both the software and test quality. On the basis of this information you can identify deficiencies of your tests and define and execute improvement activities to reduce the identified deficiencies.


You see, refering to the title of this blog post, the insertion of bugs into the software does — in a creepy way — make sense.

The results of Fault Seeding are the following two software metrics:

  • Fault Detection Rate
  • Number of Remaining Faults

The process of Fault Seeding can be split into five steps:

  • Fault Modeling
  • Fault Injection
  • Test Execution
  • Calculation
  • Interpretation


  • ISTQB/GTB Standardglossar, German Testing Board e.V., 2014
  • S. L. Pfleeger and J. M. Atlee, Software Engineering Theory and Practice, Fourth Edition. Pearson Higher Education, 2010

To Be Continued…

Fault Seeding Principles | Further development of Fault Seeding | Empirical study with a real-world project