Test Processes — Your friend, not an enemy
“Quality means doing it right when no one is looking ~Henry Ford”
Let me start by sharing my experience, which I faced a few months back. We received a bug that required only a minor fix. The code changes had been made, no issues with the build, unit tests, and everything worked properly but no local testing had been done.
I was engaged in testing. I verified the fix and it was not working as expected. Worked with the developer and informed that the fix was not working. We hopped on a call and began investigating the root cause.
The developer said, Ah, I missed fixing the changes in the other module. Let me fix it. Deployed the changes again, started testing the scenarios, and again it didn’t work as expected. The developer double-checked and said, “I know why it’s failing; please allow me to fix it.”
Once again, changes were deployed, and the fix worked as intended this time. (It took almost 4 to 5 hours to get it done).
Instead of bouncing back and forth, we could have saved a few hours if the developer had tested the modifications locally. Getting the build artifacts and upgrading the environment is time-consuming.
Your code changes may be minor or critical fix, but it requires thorough testing. Having a process for testing does not imply that you are a bad developer. It is a way of showing how well you are concerned about the product you are working with.
My code is perfect and I don’t need to waste my time on creating an environment, and deploying the changes to test it.
Even the most experienced professional can make a mistake. It is always good to do a round of testing rather than to deal with an issue or escalation later. Nowadays, most of the code is automated. Create an environment, deploy your changes, and do all the required setup. Carryout the testing yourself or involve the quality engineers in your team and explain the use cases so they can run the automation on the environment.
Framework and tools
Computer Automated Test System (CATs) is a software testing and analyzing tool used for web & mobile web applications testing; it supports iOS, Android, and any HTML5 browser. CATS can generate the results in different formats as a text file, generate different graphs, send the execution results to mail, and allow printing the execution results. CATS can run a very large set of test cases and generate the results. We are automating the test scenarios with the below-listed tools.
- SeeTest for iOS
- Appium for Android
- Ruby
- Git
Using Jenkins to run the automation scenarios by injecting the environment variables.
Analyze the test results
Once the job completes, we collect the test results, and export them to a results parsing server, which shows the results in the below captured snapshot
Verify that the test results are good and all the scenarios are working as expected after the automation run is completed.
If any test scenario fails, investigate why the scenario is failed. Is it due to an intermittent issue? Bad data? or is it a result of a recent code fix?.
Check the logs to determine the root cause and, if necessary, engage the developer to change the code. Once the patch is finished, deploy the changes and rerun the test to check the failed case is no longer failing.
We have identified many bugs with our automation framework.
Conclusion
Always consider testing your changes even if it’s just a few lines of code changes. Code modifications should not be merged solely on an assumption or theory; instead, they should be thoroughly validated. Make better use of the test processes and automation tools we have to ensure quality.
