From Zero to Quality: an Introduction to Software Quality for Developers
Software quality and testing are sometimes neglected fields on training of new developers. This could lead to a team that cares poorly about the quality of the software they are developing, tests with low efficiency, and a lot of bugs that could be prevented before their go to production.
This post intends to guide new developers (or even old ones) to the base concepts and ideas of software quality. So they can begin to apply for themselves quality processes and ideas on the day-to-day work with their team and, also understand the importance of testing.
Why do we test?
We should begin with the most important question. Why do we even test at all? The obvious answer would be “To find bugs!”. It’s not a wrong answer, but it doesn’t contemplate all the reasons why we should test our software. More than finding bugs, testing is to find out if we are developing the right product and if the product works right. It worths nothing if we develop a perfect product without bugs, but it fails to do what our client truly needs. If a client needs a product that adds A + B and we deliver software that subtracts A — B (without bugs!) then the client will not be satisfied. Real-life requirements are lots more complex than this example and often people fail to communicate what they really want to be developed. This miscommunication could happen even between Product Owner and Developers or Bussiness Team and Product Team. Because of that, the test is also needed to verify if the product we are delivering fills the requisites that the client or product owner needs.
We test as well to gain confidence in our product quality. The more confidence we have, the more refactors and improvements the development team could do without the fear to break things leading to better software overall.
In addition, test to mitigate risks involving our software. Some software can’t fail at all (critical systems); Some can endure some types of failure. Overall we should test to prevent and understand these risks so that our software could respond better under these conditions.
Finally, we don’t test to prove the absence of bugs. It’s impossible to guarantee “bug-free software”. We test to find bugs, fix them and prevent new ones to happen, but we can’t prove that bugs will never happen. The rigor of our tests will always be determined by the criticality of our software.
A famous rule on software quality is the Rule of Ten. This rule says that the cost to fix a bug increases by 10 times for each development cycle. This means that a bug found during development would cost, for example, $10 to fix. If found after integrations would cost $100, after UI or user testing $1,000, and at production $10,000. It happens because after each cycle, more time and people need to be involved to fix and deploy the fix. Because of that, one of the principles of software quality is to find bugs fast and in the early stages, so they can be easily fixed.
What do we test?
Many factors can influence what will be tested. The more complex a feature is, the more tests we should have to guarantee the quality. This often happens in features with many paths that a user can take to complete or with features with very convoluted business rules that confuse even with people used to it. The risk involved and the security expected also influence what will be tested. Softwares and features with high risk (payment features, security, sensitive data for example) demand more tests to guarantee the quality standards. How much of the code is cover by automated test define what will be tested because if the team is more confident that a feature is covered by automation, they could focus the testing on other areas. At least, the experience of the developer and QA heavily influence the testing. More experienced developers would know which points on the software could more probably present bugs, and more experienced QAs know where bugs usually happen and then focus on testing these parts.
How do we test?
When writing a test, it must follow some guidelines so it fills its purpose. Tests are usually formed by pre-conditions, input values, expected results, and post-conditions. Pre-conditions are the state of the software must be before the test is executed. “Be logged in with an admin account” or “Product X exists on database” are examples of pre-conditions. The input values are actions and data that will be inputted into the system during the test. In consequence, the expected result is what we expect will happen after the inputs if our software is working right. And post-conditions are the state of the software after the test is finished. We then, execute the test and compare the actual with the expected result. If they are equal, then the test is green, if not we should investigate what went wrong and fix it.
We should also guarantee the quality of our tests because we want them to fulfill their purpose to test the software. To help to write good tests, its import to remember of 5 attributes a test should have:
- Effective: It should test one thing only. With that, when a test fails, we know exactly what broke.
- Direct: It should not have steps that do not directly relate to the test, so it fails only if the actions under testing fail.
- Traceable: It should be able to identify what requisite, task, or feature it’s under testing, so we know what is broken.
- Reusable: Other tests could reuse some steps or even the whole test. With that, we reduce the number of steps of the test that needs to be written.
- Self-explanatory: If another person reads your tests, he/she/they should be able to understand what is be testing, steps, and expected results.
More often than not, we have cases where it’s impossible to test everything. Especially with input fields, where we can’t test every combination possible. For example, on an age input field we can’t (and shouldn’t!) test for every natural number because it will take infinity time. It’s usually safe to assume that with a few tests, we guarantee that this works for all natural numbers. The question here is which tests do we use to guarantee maximum quality with fewer tests? To answer it, we will use 3 common techniques: Equivalence Partitioning, Boundary Values, and Pair-Wise.
Equivalence Partitioning is a technique that divides the input domain into classes of equivalent data. It means that all data inside the same partition will be treated equally by the software under testing. With this, we can assume that one test for each partition is enough to prove that all inputs inside the domain of that partition work. Let’s use again the example of an age input field. This field should only allow ages from 18 to 65 years old. We can divide our partitions as follow:
In this case, we can make 3 tests to guarantee that this field works. We select one test input from the first partition (0 to 17), one from the second partition (18 to 65), and one from the last partition (66 to infinity). With that, we can test that our input works for valid inputs (18 to 65) and invalids inputs (below 18 and above 65). In some cases, we can take two or more values for one partition if we want to test some specific cases. To choose which values from each partition, we will use a second technique called Boundary Values.
Boundary Values is a technique designed to select representative values for each partition. It is based on the hypothesis that most bugs occur on the boundary between each partition. Using our previous example, we would choose 0 and 17 from the first partition, 18 and 65 from the second, and 66, and the max value the field accepts from the third. Most bugs occur on the boundary values because they are usually corner cases on an if-else statement and should be tested more often. With both techniques that we saw combined, we reduced the number of tests from almost infinity to only 6 without losing almost any confidence in our testing.
Pair-wise is our last technique. It appears when we have multiple fields combined and it will take a ridiculous amount of tests to contemplate every possible combination. So we aggregate different combinations together, reducing the amount of testing without a big loss of confidence. For example, we have a form with 3 fields. The first field input accepts anyone letter on the Latin alphabet, the second field accepts any one-digit number, and the third accept a boolean (true or false). If we would test every possible combination, it would take 520 (26 * 10 * 2 ) tests. Instead of that, we will at least use any valid input once. This will reduce the number of tests that we need to only 26 (the biggest domain between the three inputs). Although it doesn’t guarantee the same level of confidence as testing all combinations, most of the time its enough for our software and will reduce testing time by a lot
Next Steps
In this post, I tried to cover the basics of software quality so that more developers could apply software quality techniques to improve their jobs and create a better application and improve their confidence on how is the software they are delivering. For the next posts, I will try to explain more about some processes to increase the quality of your team and about test automation.
