4 Types of Performance Tests That Uncover Critical Bottlenecks
Testing Different Types of Traffic Patterns Uncover Important Performance Data
You may have heard the old cliche, “There’s more than one way to skin a cat.”
Well, that’s gross. Why in the world would anyone skin a cat? And why were there people out there figuring out multiple ways to do it?
We propose changing the phrase to “There’s more than one way to skin your website, applications and infrastructure.” (It might take a while to catch on.)
Either way, when you’re performance testing your applications, it’s important for you to run multiple tests with various configurations — depending on what you aim to learn.
So, here’s a quick reference guide with four types of performance tests that can help you uncover all kinds of bottlenecks.
While it might look like the basic “vanilla ice cream” of the performance testing world, the load test is the perfect go-to when you want to understand your app’s performance and whether or not it’s meeting your goals.
When setting your growth and revenue goals, you know how many purchases you’re expecting per hour, or how many visitors you expect to attract, or other business-oriented goals.
Recreating those functions in a user scenario and load testing the processes to verify your app can handle the pressure is essential to meeting those goals.
A load test typically has a few ramps — meaning the number of concurrent Virtual Users you’re testing gradually increase through specific time increments — and can last anywhere from 10 minutes to a few hours.
Your load tests should be long enough for your user scenarios to iterate a few times to ensure you have valid, analyzable data.
It’s very common that our users’ applications don’t meet their performance goals, but that’s OK. That just means you will be able to identify problems with the test data and fix the issues.
If your load test completely brings your website or app crashing down, you can just consider it a stress test, which is the test configuration we’ll cover next.
Now, onto the fun stuff. How many concurrent users can your app handle before everything breaks?
Similar to load tests, you will want simulated traffic to ramp up to the eventual maximum number you want to test. Simply jumping right up to that max number is a different kind of test that teaches you different things. We’ll get into that type of test next.
If you have some historical reference in your analytics, such as the last time you had a big sale, earned some good PR, etc., take a look at how many people wandered over to your website or app last time, and use that as a guide.
If you don’t have a reference point, try running stress tests that work their way up well beyond the traffic you typically see. You know your system better than anybody, so make an educated guess on whether your app’s max capacity is 5x-, 10x-higher than normal traffic, and just run the load tests from there.
A spike test is exactly how it sounds — a pronounced spike in traffic coming to your website or app at pretty much at the same time.
We’ve worked with engineers who needed to prepare for gigantic Black Friday and Cyber Monday sales, had their products featured in Super Bowl ads, appeared on Shark Tank and countless other reasons to warrant spike tests.
The common thread here is they know there will be a captive audience seeing their product, and it’s a virtual certainty that many of them will be immediately Googling them or heading straight to their website or app, and it will happen at once.
Engineers not preparing for this kind of exposure is what usually leads to embarrassing news articles and them rushing to social media to sheepishly say, “Sorry, everyone, but so many people like us that our site crashed!”
Some companies and marketing teams try to wear this as a badge of honor publicly, but behind the scenes, we can assure you, nobody is smiling about the fact that their website is down and they’re missing out on potential revenue. Especially in some cases, where that revenue could significantly alter the long-term growth of the company.
Are your systems functioning at their max performance throughout your hours of operation? Is your application and infrastructure optimized to handle purchases and other functions 24/7?
If those are things you’re planning to achieve or looking to consistently maintain — it sounds like you’re in need of a few soak tests.
These tests uncover performance bottlenecks stemming from a system being under pressure for an extended period of time.
Issues such as memory leaks, resource leaks or corruption and degradation that occur over time can be identified and optimized based on data accumulated from soak tests.
After you’ve pinpointed your max capacity of concurrent virtual users through the other test configurations, we recommend you run your soak tests at about 80 percent of your max capacity.
Why 80 percent, you ask?
A soak test provides data on how your system reacts when pushing a high number of transactions through your system in a relatively short amount of time.
The benefit here is that it can, for example, only take a six-hour load test to simulate a 48-hour workload.
Optimizations based on stress test data gives you confidence in your application and infrastructure, which means you’ll be able to accurately forecast and set business-oriented benchmarks.