Software Testing And Testing Types/ [Software Development Engineer in Test Article Series Part 6]
This is the sixth article of my “How to Become a Software Development Engineer in Test” article series. All of these articles have been written in ‘question-answer’ format. As I learned from some of my Filipino friends, this method was developed by Socrates to explain things better. So I hope the techy and non-techy guys will benefit from these articles.
You can find the previous articles here. It is highly recommended to read the previous articles before this one.
Since I have been a software developer for more than 11 years I will provide some sample codes of real and working examples. These code samples originated from mobile applications, desktop applications, web applications, web services and various database procedures as well.
To explain things much better, LadyQ and SorBi are intensively used in my articles as samples.
Lady Q: A businesswoman and client for the SorBi app. She pays money to a software development company to have the SorBi app developed.
SorBi: An android application for creating and sharing polls. SorBi is used as an example in this article series. This app is available on Google Play and it supports English and Turkish languages.
Let’s enjoy learning the exciting stuff of Software Testing!
Question: What if Lady Q finds problems about her Android app SorBi?
Answer: Firstly, we must clarify the word “problem”. There must be common terms perceived alike by the customer, business team, and development team. That’s why we follow a software development methodology and an implementation of it like Scrum. Let’s talk about things in a more definite meaning.
Q: Okay, then, for example, Lady Q wanted a “only members can comment” functionality. But the SorBi app does not require membership for comment. What to do next? Who is to blame?
A: As it is indicated in our first and fourth session, the development team develops the app based on the client’s needs. If requirements are not clean enough, such problems can happen. In order to avoid such situations there is a testing mechanism in Software Development Life Cycle (SDLC).
Software Testing is a continuing process to check whether the software matches the requirements of the client.
Even the Testing phase is the 4th step of SDLC, a lot of different types of testing is performed through every step of the SDLC. Infact, in Agile, testing is mostly a parallel process happening at the same time with other processes.
The problem, you mentioned above, most probably originated from a fault in requirement gathering or wrong documentation. So, testing activity should start at the very beginning of the SDLC process.
Q: I see, Testers check all the processes of the SDLC from the beginning?
A: Not exactly. I mean, testing is applied to the whole SDLC, but the performers change at every step. In Agile&Scrum Framework the responsibilities of every action, in different steps of SDLC, belong to different stuff. Since each action/process has its own characteristics, testing of each action differs from each other, so as the testers. To sum up, tests differ in every step as well as testers.
Q: As everybody knows, the first step of SDLC is Requirement Gathering. But, there is no app yet. So, what to test, how to test, who to test?
A: It seems there is nothing to test. But, indeed, it is the most important step to perform testing. If you wrongly match the first button of your shirt, the rest of all buttons will be wrongly matched. So you have to go back to the first button to button up your shirt correctly.
It is time to introduce the term Bug or Defect. A bug (defect) is something that the software doesn’t do that the product specification says it should do. So as you see from the sample, it is much more costly to fix the bug if it is detected in the last steps of the SDLC. It is easier to fix the bug in the early stages, so is the cost, effort and time aspects.
Testing activity should start with testing the requirement against good requirement characteristics. Good requirements must meet the SMART criteria; (1) SPECIFIC, (2) MEASURABLE, (3) ATTAINABLE, (4) REALISTIC, (5) TESTABLE.
In the first step (Requirement Gathering) of SDLC, a Software Requirement Specification (SRS) Document is created by the Business Team. In the second step (Design) of SDLC, a Software Design Specification (SDS) Document is created by the design team. In the 4th stage (Test) of SDLC Test Plan Document is created mostly by the QA-Lead. These documents are tested/checked by individual review, peer review and group review. Document testing, reviewing process is called Static Testing or Verification Process. It only focuses on testing the documents, not the software. Static Testing is done manually, not automatically.
Q: But there might be a problem from the coding phase. How to check that process?
A: Developers perform Unit Tests and Integration Tests at the 3rd step of the SDLC; Coding. Since the developers know the actual code, that kind of testing is also called White-Box Test. Unit Test (1/4 of Dynamic Tests) is testing the smallest parts of the code, like testing classes and methods. Integration Test (2/4 of Dynamic Tests) is testing the different functionalities’ interactions and consistency, like integration between the Sign Up Module and the Purchase Module.
Since Unit and Integration Tests are performed directly on the software itself, they are part of the Dynamic Test. Dynamic tests are performed directly on the software. Another name of Dynamic Test is Validation Process.
Q: I am getting confused. If every developer tests his/her code, then why do we have a Test phase in SDLC?
A: That’s again a critical point buddy. You’re touching the most important points. In the 4th step of SDLC, software testers perform System Test (3/4 of Dynamic Tests). That’s where software testers come into the scene.
System Testing is a kind of Dynamic Test, because it is performed over software.
Since the testers do not know the underlying code of the software they are testing, we call it Black-Box Test. If they know something about the code behind, then we call it Gray-Box Test.
System Test is divided in 2 parts; Functional Test, Non-Functional Test.
Functional Test consists of Front-end Test, Database Test, API Test (we saw these 3 layers in the last two sessions), Smoke Test and Regression Test. Functional Tests are performed by QAs (Quality Assurance Engineers).
Non-Functional Test consists of Performance Test, Security Test, Load Test, Stress Test. These tests are performed by Performance Testers.
Q: I am really shocked by a lot of testing types. How much is necessary to perform these tests?
A: Let me ask a question.
How to assure the quality of a software?
Correct answer is: By performing software tests.
The main goal is to meet the customer’s expectations.
So make sure the software meets the customer’s expectations.
Quality assurance is a way of preventing mistakes and defects.
Since a long time there have been manual testers to perform Functional Tests. But the Automated testers had a game changing effect. They are also called Software Developer Engineer in Test (SDET).
SDETs use software frameworks and coding ability to perform automated tests, while Manual Testers test the software by directly interacting with it as an end user. So SDETs perform tests faster than Manual Testers do.
To sum up, there are tests to be done manually, most tests are done automatically.
Q: It is getting clearer, but I have a question. In the Scrum team, SDETs do tests directly over the software, because they perform Dynamic Tests. If there is something changed by developers in the code behind or in the database, won’t it be chaos? Moreover, nobody can solve the problem, even detect the problem.
A: That’s why all tests are performed at different stages/environments.
Q: What is an environment? What is it used for?
A: Environment is nothing more than the software with all it’s three layers (Frontend, API, Backend).
Think about a simple website that has only 6 web pages and it doesn’t have a database and API. As we know not all software requires API and database, so our sample web site does.
“www.abc.com” has it’s content in a folder named P. Now we call P as the Production Environment. The Production Environment is accessible by end users, simply by navigating to “www.abc.com”.
While the “www.abc.com” is on the air, Developers might want to develop new functionalities. To achieve this appropriately, they create an exact copy of the current web site in another folder named D. Now, we call D as the Development Environment. The Development Environment is dedicated to developers, and it is accessible by developers simply by navigating to the assigned domain name, something like “dev-abc.com”. As you guess, folder D holds the exact content of folder P. Moreover, developers are adding new codes for new functionalities to folder D, without interfering with folder P.
After developers finish coding the new functionality, they create a new folder named T. Now we call T as the Test Environment. When developers finish a functionality they publish the software to the Test Environment. While testers are testing the finished functionalities in folder T, developers can work on new functionalities in folder D. So while developers are working in the Development Environment, testers work on the Test Environment. These are totally different folders. The Test Environment is accessible by testers simply by navigating to the assigned domain name, something like “test-abc.com” or “qa-abc.com”. Whatever the URL is, the folder that has the contents of the Test Environment is totally different from the P folder, as well as D folder. System Tests are performed in the Test Environment.
So no one can change the code in the T folder. Happy hours for testers to test the T folder freely. Actually they are testing a specific build of the software. Another build is published in the P folder, and another build is published in the D folder.
Q: What if there is a problem in the Test Environment? Let’s say testers can not reach the Test Environment by navigating to “qa-abc.com”.
A: There are some steps to perform before testers start system tests. For example checking if the web server is up or the database is accessible. There is a test called Smoke Test, performed for checking if the most crucial parts of a software are up and running.
Think about the SorBi app. As you know from the previous sessions, the SorBi has three layers; (1) Frontend, (2) API, (3) Backend (file server, database). If we want to be sure everything is fine with the SorBi system we have to test each level to verify if they are running and accessible. Moreover, we have to test the crucial functionalities (like Sign Up, Log In, Create Poll, Vote Poll etc.) of the SorBi. All these tests form the Smoke Test.
For testers, when they are about to start testing in the Test Environment, firstly, they perform a Smoke Test to check if the minimum system requirements are working well. If so they can perform the System Tests right after. If not, they inform the developers.
After publishing the software, Smoke Test is performed in the Production Environment, on daily basis, in order to verify that system is up and users can interact with the system without a problem.
Q: All these are very well till now. I wonder what if Lady Q wants to test her app with other testers that she relies on?
A: That’s what we call the User Acceptance Test (UAT). UAT (4/4 of Dynamic Tests) is performed by the client itself or by the hired manual testers and SDETs. The tests performed manually by testers, clients and end users are named UAT-Beta Test. The tests performed by automated scripts are named UAT-Alpha Test. This test is done by SDETs.
Q: In which environment do they perform UAT tests? I think it will be a chaos if they use the Production Environment or the Development Environment. Do they use the Test Environment that the testers of the Scrum team use currently?
A: Creating a new environment for a software is not a big deal. Simply, change the production path by selecting another folder and assign a new subdomain. That is all. If you have a database, create a clone of it. That’s it. If you have an Android application, create a new release with a different version number.
So, there is no need to get things messy. The appropriate solution for UAT testers is to create a new copy of software in a new folder named U. We call the U folder the Staging (Pre-Production) Environment. UAT Testers can access this environment with an associated URL, something like “uat-abc.com”.
Since UAT is performed directly over software, it is a part of the Dynamic Tests.
Q: There is a disturbing thing in my mind. It is really scary. 😨😬I am afraid to even think about it. But, okay, I gathered all my courage. Let me ask and relax at the end. So, so, so, what if a new functionality or a code update is inconsistent with other existing ones? Please say they will be nothing bad with the software, please 🤢…
A: That is also a nightmare for the development staff. But, yes, there is a solution for such issues. We call it the Regression Test.
Once a code is changed in the software, or a datatype of a column is changed in the database, or a new functionality is added to the software, or a bug is fixed etc. we test all the older functionalities to check whether the new code or change breaks the existing functionality unintentionally.
We have to check that, because all the code and system are somehow interconnected to each other.
As it is stated above the Regression Test is part of the Functional Test of the System Test, and so, it is performed in the Test Environment, mainly by SDETs. It is performed right before publishing the software.
Q: Thank you very much buddy. You saved my day. The Regression Test made me relax. Now, I can eat my lunch. Otherwise I won’t eat because of the fear of new functionality inconsistency. Thanks to the Regression Test. Will you join me for lunch?
A: Of course, I will. ✔️