Think beyond the conventional QA strategy

Software has become an essential part of human needs nowadays. We do see lot of new web and mobile apps are released to market to facilitate human activities rapidly. The question is raised in to our mind as quality assurance professionals how we met those necessary quality standards to achieve production ready software. Nevertheless, most of the IT companies are adhering to the complete test automation approach and DevOps practices in the default software release cycle. These initiatives led QA professionals to looked back and come up with comprehensive QA strategy to fit in to single or multiple projects deliveries to fulfill above requirements.

Why we need QA strategy, is the fundamental question we need to answer before start anything. If we looked closely, recent industry trends and community chase towards new automation tools and their usage mostly. I would say it’s a good sign from the technical or rather operation perspectives to bring the right values to your QA automation or nonfunctional requirements relies on project goals. Even though if those in placed we were witness that, some projects will get failed in the long run mostly due to quality related issues. That will prove the facts that there is missing piece in between what we do and how that bring the right ROI to the product or project we deliver in term of quality aspects. This is where QA strategy come in to play as a proper governing body to ensure to most appropriate QA practices will be execute at the right time.

Next question is how we developed a better QA strategy beyond the conventional way of doing things. We don’t see much articles shared with denote to this subject and I thought of sharing few important points you can reminiscence when creating custom QA strategy for any software project. Before jump in to QA strategy we should consider below pre-requisites to make the clear path to initiate the QA strategy.

Identify the nature of the projects

Entry point to the QA strategy can be diverse due to the nature of projects. Small scale project usually define strategy inside the test plan would be wise approach but when its come multiple projects which operate under single entity used to define generalize QA strategy in the top level and managed specific QA tasks via QA test plan in the project level. That will help the person to decide which items should be kept in the global context and which items need to drag in to project level in terms of strategy perspective. I am trying feed required details with refers to second approach which I mentioned above.

Holistic view of the QA strategy

Holistic view is mainly focus on common components which need to be implemented across all projects. For an example I want to define a QA strategy for multiple projects which operate under same business domain handling different business operations. By default, we need four environments (Dev, QA, Staging and production) to managed development and QA activities in the project life cycle. If management or client wanted to eliminate one environment due to cost factor in one project, then we can customize the requirements in the project level. Likewise, we can define how QA role and their responsibilities interacted with different settings and how CI & CD processes triggered in between environments and in what manner those integration points handled effectively. I would suggest to taking your idea as a design rather writing paragraphs. Then review the artifacts with the subordinate and get their feedback. Then you will get a clear picture what you are trying to do.

QA Roadmap

You will get confused at some points how I defined a roadmap even before QA strategy and test plan in place. This is where your QA team support is required. Let’s take a simple case study which I was engaged sometime back. In that case study I have broken down my QA roadmap to 3 phases. Each phase governs by 2 stages.

E.g.:

Phase 1: Assess & Define

Phase 2: Design & Execute

Phase 3: Evaluate & improve

Each stage contains one or more action items. Then I have assigned my QA team member to conduct several proofs of concept to finalize test management tool, test automation tool used across all projects to cover API, UI and mobile automation and DevOps integration to improve the release cycle. Finally, we connected all outcomes with the action items and aligned it to stages along with timeline.

Now all set to catch the big fish. We are fully confident with the direction of QA strategy and what contents need to be update in the document. As per my understanding below components should be mentioned in the QA strategy document.

Mission

QA strategy should have a clear mission. Mission statement should be simple and goal oriented. Let’s assumed if you want to achieve any project specific QA goals then you can specify it under test plan.

Mapping test levels.

Any QA activity performed under project level always based on test types. Due to that reason we should have a clear idea about the testing levels we are planning used (E.g.: API Testing) with below factors.

· Why is using (E.g.: Ensure communication between components)

· Who is using (E.g.: Automation QA)

· What is the purpose (E.g.: Monitor web services stability and data issues)

· When it is executed (E.g.: Adding parameters and new service integration)

· Where it is using (E.g.: CI and testing environments)

· How it is using (E.g.: Using automation tools like Postman, Rest client etc.)

Now you have clear picture about the test levels and how it conducted under project life cycle. Even though we are still bit confused of selecting testing levels for the project. According to my understanding it is totally depend on the project nature. That is why we used agile testing quadrant concept which use to map required test levels. Quadrant itself we can align our projects and prioritize test types in terms of ROI. Then you can map the selected test types.

Requirement grooming

Under requirement grooming phase we should identify what information should be expected by default from the user stories to the QA support activities. That will clearly help QA engineers to identify clarity of the requirements along with scope of testing.

· All user stories should contain acceptance criteria

· Each acceptance criteria should be testable

· Scope of the story should be clearly defined.

· Technical information with related to the story should be aware by the QA.

· Estimates given for a story should include the testing effort as well.

· Automation scripting effort also communicated during the planning phase.

· Test case design (Smoke, Regression & custom release suites) should be calculated under sub tasks in the story.

Agile QA process & bug transition

Agile QA process should be explained the activities which planning to execute during the sprint cycle. As an example, let’s assume we bound to sprint 01 tasks. Then we should define manual test case design, automate scripts design, test execution, defect tracking, automation QA jobs integration and monitoring parts in the process. Finally, we should have clear integration points in between those tasks along with the outcome.

Bug transition cycle should be defined in advance when you are planning to use any bug tracking tool. We all know that tools like Jira, contains default bug transition workflow. Then we can customize those workflows according to our needs.

Automation testing approach

We should address automation testing approach in side our QA strategy since it has a high demand and the direct impact to the project delivery cycle. Mike cohn’s Ideal test automation pyramid will be inherited to the project structure to get a proper understanding how we want to build the automation framework with refers to below levels.

Apart from that we should consider below best practices when attending to automation tests creation and execution

· Remove uncertainty from automation tests

· Test tools evaluation (POC) before automation

· Periodic reviews to automation tests for validity

· Decide test coverage before automating

· Automation regression tests

· Aim for fast feedback using DevOps practices

CI & CD pipeline evaluation

CI & CD pipeline QA automation should be clearly defined under automation QA strategy to bring the right infrastructure to the project to integrate the automation suites. Below is the sample delivery process we can noticed nowadays.

What are the best practices we need to adhere when integrating automation test suites with CI & CD pipeline.

· Prioritize automation test scripts along with the duration to complete pipeline.

· Defined mechanism to trigger automation jobs in each feature deployment.

· Identify the necessary infrastructure to managed build agents.

· Consider billing model attached with the build minutes when deciding pipeline.

Nonfunctional testing approach

Depend on the usage we should consider non-functional requirements to avoid any operational issues which can occur in the production ready software. These requirements will be arising on demand basis and we should have a clear approach if any of the test item is planning execute in the project life cycle.

E.g.: Performance testing, Security testing, Compatibility testing, Usability testing and data conversion

What are the best practices we can apply to make sure the way we do things right.

· Identify the scope of the nonfunctional requirements

· Justify the tool used for evaluating the criteria.

· Configure the test environment according to requirements.

· Defined the evaluation criteria for the requirements.

· Execute and improve the report outcome

Test reporting and metrics

Test reports are the final output of our QA activities. In that case we should consider comprehensive reporting mechanism to get right decisions. More over we should include the metrics which we plan to evaluate the health of the release. Below are few samples reports and metrics which we used for evaluating the software products.

Reports

· QA Release Note

· Test coverage report

Metrics

· Defect severity

· Daily Test Execution Progress, by status

· Test Executions by Test Cycle

· Test Executions by Tester

· Top Defects Impacting Testing

· Test Distribution

· Requirements to Defects traceability

Are you confident to create a comprehensive QA strategy, in your next software project? Yes, I hope so after reading this article.