Risk Based Testing : When and How?

In almost every project, development gets delayed which leads to squeeze in testing time to meet the release deadline. This means testing has to be done in limited time maintaining the same quality. Fortunately, Risk-Based Testing provides a way to de-scope tests in a logical manner and allows visibility of the risk coverage. RBT allows the stakeholders to make decisions based on QA’s assessment of risks.

Risk is the probability of occurrence of an uncertain event. It could be events that have occurred in the past or current events or something that could happen in the future. These uncertain events can have an impact on the cost, business, technical and quality targets of a project.

Types of Risks

Level 5: System crash, loss of data (ex. leaving name field blank is crashing the app)

Level 4 : Boundary conditions (ex. App is not working fine when name has 20 characters)

Level 3 : Negative Testing scenarios (ex. App is not working fine when name has numbers or special characters)

Level 2 : Inconsistent behavior (ex. App hangs for 10 seconds on entering username, but the frequency of the bug is 1/50)

Level 1 — Spelling mistakes/UI issue

Chances of fixing the bugs just before the deadline is rare. If it is a Blocker Bug, the probability of occurrence would be <1% , and it’ll be fixed. If it is a Medium or low priority bug, the probability would be <5% and it won’t be fixed just 1 day before release.

RBT — When to do it?

1. When QA team is out of time
2. The project is new and team does not have experience
3. When team is following Incremental and iterative models

RBT — How to do it?

Here are some ways to do Risk based testing:

Prioritization

Testing is always a sample. It is impossible to test all the possible combinations. So, when time is limited, QA team needs to prioritize the test cases.

How to prioritize test cases?

• Which functions and attributes are critical
• How often a feature is used
• How visible the problem is to the end user

You should make a risk calculation table like this, assigning weight to different metrics and then calculating the sum to access the risk

The formula to calculate the risk is: R(f) = P(f) * C(f)

R(f) : Calculated risk of function f

P(f): Probability of fault in functin f

C(f): Cost related to a fault in function f

Automation & UT

Another important thing that takes up most of the testing time is Regression testing. The solution is Automation. Automation saves lot of time and gives a clear view of product’s stability.

Unit Testing : Unit tests should be automated as well and rerun every time units are changed or integrated.

Automation : The solution of Regression Testing

Documentation : Yes/No?

Proper documentation of test cases is often not needed. In agile approach, it can be avoided to save the time. Just brief summary or pointers are fine to execute test cases by any tester.

QA Skills

A lot of time can be saved if QA team has good experience & skills. A good QA can write better test cases which have higher chances of finding bugs.

Bug Resolution

Another important question is which bug to fix when the deadline is near. Here are some industry guidelines to do that.

Rule 1: Fix only important defects.

Rule 2: Change requests and small defects should be moved to the next release.

Rule 3: Correct defects in groups, only after blocking failures are found.

Rule 4: Use an automated “smoke test” to test any corrections immediately.

RBT Benefits

For Management:

1. To see progress in terms of risks addressed and benefits that are available for delivery
2. To manage the risks that block acceptance
3. To make the release decision.

For Testers

1. Approval to test against risks in scope
2. Clearer test objectives upon which to design tests

Thank you so much for your time.Your comments and suggestions are welcome!

Happy Testing!!