How to identify and mitigate software testing risks!

Image source: Dreamstime.com

“To err is normal, but uncertainty is not welcome,” write Tom DeMarco and Timothy Lister in their book Waltzing with Bears: Managing Risk on Software Projects. This expression perfectly describes risk management in software projects. But what are these risks, and why do you need to face them?

Brief Overview:

Risk management is the process of identifying, assessing, and managing risks. It is performed in both the planning and execution phases. An effective risk management strategy and its application drastically reduce the chances of execution failures in software development. In this blog, we are going to cover various possible risks for any project, ways to treat the identified risks, impact analysis for multiple types of risks, risk mitigation, and role-wise responsibilities of the testing team to identify and assess risks in time.

Software Testing Risks:

Source: Tutorialspoint.com

Risk is the future of uncertainty among events with a probability of occurrence and potential for loss. To protect business interests and the quality of software applications, the QA team must be able to quickly and accurately identify and manage software testing risks. The main causes of software testing risks are:

• Incorrect Estimations/planning: Sometimes we cannot identify the exact time or resources required, resulting in inefficient task distribution.
• Constantly changing customer requirements: We all have been there, right? Customers often change requirements on the go which causes delays and more room for errors.
• Vaguely Documented Requirements: As a Tester, we always need to have a source of truth, however, it’s not easy to get precise requirements, and sometimes requirements are not properly listed.
• Uncertain Circumstances: Many times resources are on leave for some unavoidable personal work and it hampers the estimations, finding a replacement immediately is also not feasible. Sometimes we face technical issues or server-related issues, which hampers the sprint work.

Treating Identified Risks :

At times risk is identified after the fact. When risk happens despite upfront assessments, it can be treated in one of four ways:

1) Avoid or withdraw

Avoid or withdraw means, In a critical situation, do not undertake that activity that carries risk.

• A retailer that decides not to collect and store personal customer data for analysis because it cannot meet data privacy regulations is also following a risk avoidance strategy.

What will be the implication of this strategy?

• The exposure to risk will be avoided, altogether, so there will be zero risk
• The loss of business opportunities is damaging to the organization.
• All business organizations need a profit for sustaining themselves; hence if we do not undertake activities that carry risk, then there will be a loss of potential profit.
• Avoidance is maybe a costly strategy. Therefore this should be one of the least preferred ways.

2) Reduce or optimize

Reducing or optimizing means, reducing the severity of the impact.

The impact of undertaking an activity can be positive, or it can be harmful and risk is associated with a negative impact. So, In this strategy, the focus is on how to reduce that negative impact.

These can be done by optimizing the risk. So, this risk optimization can be done by balancing the negative impact and the benefits available from undertaking that activity.

• Area of Business criticality of the functionality.
• Most used features and important functionality.
• Defect prone areas
• Functionality bearing the safety and security impact.
• Area of Complex Design and Architecture.
• Changes made from previous versions.

3) Share or transfer

Sharing or transferring means, sharing the burden of loss or benefit of gain from risk with other parties.

For example:

Insurance policies can be purchased from available marketplaces for businesses and individuals. When an entity buys an insurance policy, they are outsourcing its risk to that insurance company. The insurance company will then charge a monthly premium or monthly fee for sharing their burden of the risk.

4) Accept or retain

When the cost of management of risk through other options is very high, or it restricts the achievement of organizational goals then risk retention will be the best strategy. Once it is decided to retain the risk, then the decision will be taken to manage the risk.

It is a good strategy where the chances of very large loss are too small and the cost to share or transfer the risk is very high. Here, Risk acceptance or retention helps in the growth of business and profits. You can also use another strategy.

In conclusion, neither one is the best strategy, so these strategies have to be used through a mix.

Dimensions of Risks :

There are two dimensions of risks that we should know.

• Probability — Risk is always a possibility. The likelihood of risk is always between 0 % to 100 %. The probability can never be 0%; otherwise, risk will never occur. It can never be 100%; otherwise, it’s not a risk; it is a certainty.
• Impact — Risk by its very nature has a negative impact. However, the size of the impact varies from one risk to another. We need to determine the impact on the project if the risk occurs.

R: Risk Id, P: Probability, I: Impact

R1. Late submission of information, and delays in document approval by the customer.

P: Medium, I: High

R2. Incorrect or Incomplete stated requirements.

P: High, I: High

R3. Changes in the requirement during development.

P: High, I: High

R4. Problems with the delivery of the product into production because of the unavailability of the servers.

P: High, I: Medium

R5. Problems of integration with internal systems of customers.

P: Medium, I: Medium

R6. Tight time limits influence the testing flow.

P: Medium, I: High

Risks Mitigation :

Source: Softwaretestinghelp.com

To mitigate risks and lower their impact on the system, we need to be mindful of a few things. Different risks will require different ways of mitigation:

Mitigation plan for risks listed above:

R.1) Late Submission :

• Compliance with the rules of planning and organizing meetings.
• Timely information about the unavailability of employees. (Due to vacation, illness, etc.)
• Scheduling of the meeting and the provision of the necessary information in advance.
• Some buffer has been added to the schedule for contingencies, although not as much as best practices advise.

R.2) Incomplete requirements :

• Splitting the development into short iterations.
• Frequent demonstrations of new functionalities.

R.3)Change in Requirements :

• Fixing the basic list of requirements in the contract.
• A dedicated product owner from the customer for clarification on any queries/doubts.
• Frequent demonstrations of new functionalities.

R.4) Unavailability of Servers :

• Get in touch with the customer’s IT department to fetch further details or identify the gaps as soon as possible.

R.5) Integartion issue with the customer’s internal systems:

• Communicate with the stakeholder responsible for the provision and operation of appropriate interfaces. In order to agree with all the features and performance bottlenecks in advance.
• Trying to resolve all the integration-related issues before going live.
• Installing the beta version of the product and testing the product in order to identify the problems associated with integration.

R.6) Non-availability of Independent Test environment and accessibility :

Due to the non-availability of the environment, the schedule gets impacted and will lead to a delayed start of Test execution.

• Discuss it with the project team prior to sprint planning to avoid any rework.
• The number of test users should be clearly communicated and set up, during the test planning phase.
• Risks and concerns should be highlighted to the project team as not having an independent test environment may lead to a number of issues.

R.7) Tight time limits:

• Following the development schedule.
• Timely notification of potential problems or shifts in the schedule.

Checklist for Risks Based Testing:

• Involve stakeholders in every step of the risk management process.
• Build a strong risk culture in the company; this includes attitudes, values, and beliefs. The importance of risk awareness should be instilled in the employees so that everyone is prepared.
• Communicate risks throughout your company. All departments should monitor high-value risks.
• Clearly document the company’s risk management policy. It should be further communicated to the employees.
• Clear risk monitoring processes must be in place.
• Functionalities with the Highest safety impact on the project.
• Functionality with the Highest financial impact.
• Complex area of the project.
• Error-prone code areas need to be noted separately.
• Prime factors or issues of Similar/related projects that had a huge impact on the operation and maintenance expenses.
• Situations or problems that would cause persistent customer service complaints.
• An optimal set of tests that can maximize the risk coverage
• Features or functionalities were added to the product design at the last minute.

Role Wise Responsibilities of QA Team:

Being on the testing front, it is our responsibility to detect the project risks and minimize their appearance. We need to identify, list down and then prioritize risks.

As a QA Lead:

Risks can be associated with Development as well as the quality assurance process. While working on any project, the QA lead should track risks during the entire testing process. One needs to focus on the progress of both developers and testers. Preparing a report where the time required to complete the tasks is tracked along with the percentage of work done.

As a QA Tester:

Testers play a vital role in successfully delivering any project. If a tester faces any major blockers or unforeseen situations; the project lead needs to be informed in order to eliminate the risk in time.

A few points to note:

1. The sooner risk management starts in a QA project planning phase, the better.
2. Of all 3 steps, Risk Identification is the most important. If anything is not listed and considered for further steps, the risk goes unhandled.
3. Try to find an ideal time frame for this activity. Remember, too much planning leaves too little time for doing.
4. Also, after the risk management process, if a new situation comes up, the risk management plan can be altered or updated to reflect the most current condition.
5. Historical data can be very useful for the success of this process.

Hope you find this helpful! If you do, please share with your other QA friends!