Delayed Build: How to Fit Two Gallons of Testing into a One-Gallon Jug?

Photo by Luke Chesser on Unsplash

Often, developers cannot foresee every obstacle on the way to deliver on time a product to the QA team for testing purposes. And that’s fine. We are all flaky at estimation but don’t expect a miracle situation when the QA team can test a one-week scope in one day or less. It’s impossible to do with an initially agreed high-quality level.

Once the testing time drastically decreases due to build delivery delays, the hopes of providing a good quality product are melting down. The risks arise insanely right before the release date. The thing is, the company cannot afford to postpone the release date. In that case, the company might lose a lot of money. But it depends. In any case, I see this picture over and over again, release after release.

Source: HeartyPsych

Imaging the case, when we need to deliver one feature in a two-week sprint. Assume, we even have all the requirements, and the development team can start to code from Monday. Meanwhile, QA can spend this time preparing all test data and test artifacts (create test cases and/or checklists, traceability matrix, etc). Don’t forget, the team has plenty of activities related to meetings and any kind of discussions.

And here we are at the second week of the sprint on a beautiful Tuesday morning, expecting the build will be delivered after lunchtime. But it’s not happening. No matter what is the reason, the QA team didn’t get the build on time to start their testing. Instead, they receive the build with a delay. The second week Wednesday afternoon is the time when the QA team begins testing. We need to deploy it to production on Friday.

So, we need to be on time with passing all test cases related to the new feature and go through the regression suite, execute the build verification process after bug fixes. Pretty intense! The estimates provided here are just an example. So the main goal here is to basically fit two gallons of water into a one-gallon jug. How? Well, you are smart, figure it out. We need to ship the build to production on Friday. Friday! The favorite deployment day of all team members! Am I right? ^^

Source: diyhorseownership

The “Available time for testing” definition changes, and not in a good way. No matter how much time you put into account during planning. It is always a problem to get the build on time to be able to have proper testing before the release date. After all, QAs might have their own unpredictabilities. But who cares?

When we look at the testing process from the test cases perspective and time allotted for testing there are different approaches that come in handy to form a test suite. I’m going to describe a few approaches based on my personal observations.

Approach #1: Priority based

1. Critical and major priority checks. Applicable when the team has very little time.
2. Critical, major, and medium priority checks. Applicable when the team has quite enough time for testing.
3. Critical and major, medium and low priority checks. Applicable when the team has plenty of time for testing. The case that never happens in the real life.

Approach #2: Context based

1. Compose a test suite with critical test cases and with affected functionality test cases (in spite of the affected functionality test cases priority). There might be a situation where you omit the major check that potentially had small or no changes. But you include medium and minor checks as part of affected functionality testing in a way that they have a risk to be more ‘buggy’. I prefer this approach.
2. Make adjustments that potentially lead to unpredictable events which can change the initial testing time. Potential unpredictable events:

• QA gets sick;
• no internet connection in the office;
• lousy at time estimation in general;
• developers provided a build with a huge delay;
• someone brought a dog to the office;
• etc.

QA: We have one week for testing. Great!

PM: Plans have changed. You have 3 hours.

Build delivery delays consequences:

• production bugs rate arises as the time allotted for testing was ridiculously little;
• unhappy customers;
• QA burnout (the team is in constant anxiety when it comes to testing in a time-restricted mode).

In case of fire, evacuate women, children, and elders. You have to be extremely selective in what test cases should be executed during regression.

Photo by Issy Bailey on Unsplash

Let me know if you have the same struggles on your projects. Did you come with any solution for how to deal with those delays and the testing time deficiency?