How to improve QA without increasing automated tests

Sometimes we are focused too much on automating our manual tests and forget to actually improve our QA testing processes which can surprisingly aid in improving the quality of our software too.

When there is a new feature ready for testing, many times we don’t have specific automated tests to address the new feature. Even if we did, not everything can be automated in time or due to some other constraints.

In this case, QA (or the engineering team in general) has to be able to test more smartly and efficiently in order to catch bugs and ensure a quality product is being deployed and shipped to production.

A while back a few changes were implemented in our engineering team’s testing processes to help ensure more confidence in our product post-testing. These changes have been categorized and outlined below:

Product Owner:

1. Identify big-ticket items: During Sprint Planning, identify big-ticket items. This will help the entire team be aware of the big-ticket items that need extra attention in the entire SDLC, including the testing efforts.
2. Thorough Ticket Details: Make sure that big tickets have a full and thorough written description in plain English that is conveyed from the stakeholder and/or developer (filled out in Jira or whatever project planning tool your team uses). This will help in development and testing efforts and reduce confusion about the feature.
3. Corresponding QA Ticket: Make a QA ticket that corresponds to the development ticket and is linked to said development ticket (this can be done by QA too). This will increase visibility, accountability and tracking of all QA efforts.
4. Labels: Use labels in Github and Jira religiously. These can help to keep track of relevant tickets, bugs, branches, etc for a given feature. A simple thing but can help in keeping all the relevant items together ensuring nothing is forgotten.
5. SDLC Workflow: Add custom flows to your Jira workflow such as: “dev has approved of qa testing”, “get +1 for PR”, etc. This will document exactly what stage of the cycle the ticket is in, there will no question what has been done or not done.

Developer:

1. Proposed Solution: Developer should write out their proposed solution to the ticket before writing code, so stakeholder can approve of the proposed solution and understanding of the ticket
2. Code Review: Tech lead should review and +1 the PR (pull request) before shipping/deploy
3. Lunch & Learn: Maybe have a lunch and learn session to review code w/entire dev team (if the feature is a big enough deal)

These three things can help overall in getting a feature that is built per requirements and as close to bug-free as possible, and in the end, make testing a lot easier.

QA Engineer:

1. Understand Feature: QA Engineer should discuss with the developer to fully understand the feature under test. This will help the QA Engineer to develop proper test cases to test the feature appropriately.
2. Test Matrix: Create a test matrix after discussing w/developer. I’ve always found a simple Google Spreadsheet to be a great resource for this. A test matrix can give a bird’s eye overview of what the testing will cover and can easily be scanned by anyone on the team to assess for gaps.
3. Dev Review: Get the developer to review test cases covered during testing to ensure nothing was missed. This is always nice because to get another set of eyes (from the dev who built the feature) is invaluable in helping to make sure all test cases are covered.
4. Document Results: When signing off on a ticket, leave a link to the test cases for posterity. This will make sure everyone on the team is aware that this round of testing is done and they all have access to the test cases and results that were executed.
5. Post QA pre-deploy meeting: (slack, hipchat, email doesn’t have to be in person) with relevant parties (stakeholder, dev, QA, etc.) to review what was tested, what bugs were found, known bugs, etc. This is to ensure everyone on the team is on the same page and nothing has been missed and or overlooked.

DevOps:

1. Deployment Time: Do not deploy high impact items to production right before a weekend or holiday. When this is done if something breaks then it is more difficult to get things done and or fixed due to the fact that most people will not be accessible. Thus it is preferable to deploy to production outside these times.