Myths and realities of performance testing

When we think about software testing, the first thing that usually comes to our mind is functional testing. It is understandable why we want to make sure the application or system we implement functions properly, as this is our primary objective. But that’s not all. The performance of the application should be checked in addition to testing its basic features.

Why don’t we test software performance?

Why don’t we conduct performance testing regularly, or why do we only begin testing performance when a significant performance failure takes place?

First of all, the reason why we put so much pressure on functional testing in the first place is because we assume that the use of only this type of testing is satisfactory to ensure the high quality of the system. This is, of course, a wrong assumption and applies not only to performance tests, but also to other types of non-functional tests — we often forget about security tests or usability tests of our applications, which are also extremely important.

Secondly, a fairly common fact that causes us not to implement performance tests is the lack of necessary experts and tool knowledge. Some time ago, in many companies and IT organizations, dedicated testers were mostly responsible for creating and executing tests. This is even more important for performance tests, which are usually implemented and executed by people specialized strictly in performance tests, with comprehensive knowledge and skills in using complex tools and test execution techniques. As a result, performance testing was largely reserved for a very small group of specialized IT employees. This is one of the reasons why performance tests are not implemented, even if most IT professionals see the benefits of adding performance tests to their project scope.

Implementing and executing performance tests is also a difficult profession that requires a lot of experience. One of the challenges you have to face is the difficulty of simulating the users’ traffic as realistically as possible using a test scenario, which most often consists of a series of API requests. Choosing the right environment and interpreting performance test results is also a common problem. All these challenges do not encourage development team members to execute performance tests.

Sometimes we can also think that our application does not need performance tests at all, because we wrongly assume that the load will not be high enough to cause any trouble. However, it’s common for our expectations to fall short, so to minimize this risk, we should accurately predict the traffic of potential users and conduct performance tests as well as optimize the application’s operation as needed to ensure its success.

Myth 1: Performance testing can be costly, and it may not generate any income?

Performance testing can be a time-consuming and resource-intensive process, and some businesses may view it as unprofitable because they may not see an immediate return on investment. However, it is important to remember that the primary purpose of performance testing is to identify and fix performance issues before they affect users, which can ultimately result in cost savings.

In addition, neglecting performance testing can lead to decreased user satisfaction, increased downtime, and a decrease in productivity. These negative consequences can have a significant impact on a business’s bottom line, especially if the performance issues are not detected until after the product is released to the market.

Let’s look at the list of the benefits of performance testing, which often outweigh the costs and provide a positive Return On Investment (ROI):

· Improved User Experience: Performance testing can help ensure that the application is fast, responsive, and provides a good user experience. This can result in increased customer satisfaction and loyalty.

· Better Resource Utilization: Performance testing can help identify and resolve bottlenecks, making the application more efficient and reducing resource utilization.

· Increased Revenue: A well-performing application can increase revenue by improving the user experience, reducing downtime, and improving customer satisfaction.

· Reduced Maintenance Costs: Performance testing can help identify and resolve performance issues early, reducing the need for future maintenance and debugging.

In conclusion, while performance testing may require significant investment, it is a crucial step in the development process that can result in significant cost savings and improved user satisfaction over the long term.

Myth 2: Performance testing is only done at the end of development?

Performance testing should be done throughout the development process, not just at the end. Including performance testing as part of the testing strategy and planning for it early on in the development process is essential. Doing so can help to spot and fix performance issues before they turn into more serious problems, and make sure the system satisfies performance requirements.

Shift-left performance testing is a software development approach that prioritizes performance testing earlier in the development process, rather than waiting until later stages such as production.

One of the available tools for shift-left performance testing is k6. It enables early and frequent performance testing, offers comprehensive performance test results, and seamlessly integrates into the development cycle.

Overall, shift-left performance testing can help ensure that performance requirements are met, improve the overall performance of the application, and reduce the risk of performance-related issues in the future.

Myth 3: Performance testing is not necessary for small applications?

Performance testing is important for all applications, regardless of size. Even small applications can benefit from performance testing, as it can help ensure that the application is responsive, efficient, and meets performance requirements. Additionally, small applications can sometimes grow over time and become more complex, so it’s important to establish good performance testing practices early on to help ensure the application continues to perform well as it evolves.