Test Planning and Estimation

Testing can reveal many kinds of failures but can never completely identify all the defects within software. So we need to check and verify as much as possible. Thus, a well test plan shall be created with a well estimation!. Overall, the scope of this article explains the definition and activities of test planning and how to make a proper estimation. But first let us start by talking through test planning.

What is Test planning?

Test planning is the most important activity undertaken by a test leader in any test project. So you should ensure that there’s a list of tasks and milestones in a baseline plan to track progress. Test planning is used in implementation and development projects as well as maintenance (change and fix) activities. Planning may be documented in a master test plan and in separate test plans for test levels such as system testing.

Planning is influenced by the test policy of the organization, the scope of testing, objectives, risks, constraints, criticality, testability and the availability of resources. So as a conclusion, Test planning is a continuous activity and is performed in all life cycle processes and activities. Feedback from test activities is used to recognize changing risks so that planning can be adjusted.

Test planning activities for an entire system may include:

• Determining the scope and risks and identifying the objectives of testing.
• Defining the overall approach of testing, including the definition of the test levels, entry and exit criteria.
• Integrating and coordinating the testing activities into the software life cycle activities.
• Making decisions about what to test, what roles will perform the test activities, how the test activities should be done, and how the test results will be evaluated.
• Scheduling test analysis and design activities.
• Scheduling test implementation, execution and evaluation.
• Assigning resources for the different activities defined.
• Defining the amount, level of detail, structure and templates for the test documentation.
• Selecting metrics for monitoring and controlling test preparation and execution, defect resolution and risk issues.
• Setting the level of detail for test procedures in order to provide enough information to support reproducible test preparation and execution.

Entry Criteria:

Entry criteria define when to start testing such as at the beginning of a test level or when a set of tests is ready for execution.

Typically entry criteria may cover the following:

• Test environment availability and readiness.
• Test tool readiness in the test environment.
• Testable code availability.
• Test data availability.

Exit Criteria:

Exit criteria define when to stop testing such as at the end of a test level or when a set of tests has achieved specific goal.

Typically exit criteria may cover the following:

• Estimates of defect density or reliability measures.
• Cost.
• Residual risks, such as defects not fixed or lack of test coverage in certain areas.
• Thoroughness measures, such as coverage of code, functionality or risk.

Test Estimation:

“Monitoring of testing progress is more meaningful and motivating if the test progress was based on accurate estimates under normal or predicted circumstances.”

Wayne Mallinson, chairman of Test and Data Services

What is an Estimate?

An estimate is a prediction, it could be an approximation of what it would cost, or a rough idea how long a task would take to complete.

How to estimate ?

Better testing estimates lead to better planning, execution and monitoring of tasks under a test manager`s attention. Better testing estimates allow for more accurate scheduling, realize results more confidently, and form the foundation of good management credibility.

Good testing estimates and negotiation skills will improve the working conditions of the test manager as other stakeholders learn by experience to respect the manager`s estimates.

Two approaches for the estimation of test effort are:

• The metrics-based approach: estimating the testing effort based on metrics of former or similar projects or based on typical values.
• The expert-based approach: estimating the tasks based on estimates made by the owner of the tasks or by experts.

Once the test effort is estimated, resources can be identified and a schedule can be drawn up.The testing effort may depend on a number of factors, including:

• Characteristics of the product:

The quality of the specification and other information used for test models, the size of the product, the complexity of the problem domain, the requirements for reliability and security, and the requirements for documentation.

• Characteristics of the development process:

The stability of the organization, tools used, test process, skills of the people involved, and time pressure.

• The outcome of testing:

The number of defects and the amount of rework required.

Importance of Estimate:

A study conducted by KPMG Pete Marwick found these causes of software runaways:

• Project objectives are not fully Specified (51%).
• Bad planning and estimating (48%)
• Technology new to the organization (45%)
• Inadequate/No project management methodology (42%).
• Insufficient senior staff on the team (42%).
• Poor performance by suppliers of HW/SW (42%).

• Estimates too low → planning inefficiencies which drive costs up.
• Estimates too high → planning is more accurate which drives actual costs.

Good Estimation Practices:

1- Avoid off-the cuff estimates — pressure from management.

2-Allow time for estimate and plan for it.

3- Use historical data and developer -based estimates.

4- Estimate at a low level of detail.

5- Use several techniques (ex: LOC “ Line of Code”, Function-points).

Conclusion:

Test estimation is an essential part of any project, and should be done as early as possible in the project. It sets realistic targets (budget, staffing and time) for the team. The testing approach strongly influences test planning and therefore the associated estimation of testing.