Short circuiting method executions to assess test quality

Short-circuiting method execution

Example of a test and a pseudo-tested method

Role of short-circuiting for test improvement

  • short circuiting the complete execution of methods provides valuable feedback to developers. The developers have clear goal to write a test: to make this method not pseudo-tested anymore. Developers are more comfortable reasoning at the granularity of a method than at the statement level (fine grained traditional mutation testing).
  • short circuiting methods has revealed the presence of pseudo-tested methods in all the projects that we have analyzed, even the ones with very high code coverage (cf. picture below). Development teams of all Java projects can benefit from this type of analysis to assess their test suites and improve them.
  • interviews with developers reveal that some pseudo-tested methods actually reveal major weaknesses in the test suite. We have collected empirical evidence of test suites fixed after running a short-circuiting experiment.
21 open source projects that all contain some pseudo-tested methods

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Benoit Baudry

Benoit Baudry

More from Medium

Test Automation as a Zero Downtime Deployment Supporting Technique

Azure AD Integration in Spring Boot Application

Azure DevOps Introduction — Create test plans (part 4)

Azure DevOps CI/CD 101 | Continuous Integration and Continuous Delivery