Seven Best Practices for Load Testing Your E-Commerce Website
Optimize your website for peak performance and load times
This is the busiest time of year for a lot of companies, and it’s an especially busy time if you’re an “e-commerce” company.
Note: We put e-commerce in quotes in that first paragraph because pretty much every company sells something online these days, but we’re referring specifically to online marketplaces and companies who only make money from their websites and apps
Whether you’re preparing for the imminent Black Friday and Cyber Monday deals, or you’re just a super-smart engineer who is continuously load testing — these seven tips could help you lower bounce rate, increase engagement and ultimately boost revenue.
Here are some proven tips to help you get the most actionable data from your tests.
1. Use Data Stores for Realistic Tests
To make your load tests more realistic, you’ll want to include randomized data. But you don’t need to hand-enter data into a user scenario script. Instead, just use Data Stores, which contain parameterized data created from a CSV file.
Here’s more information from the Load Impact Knowledge Base, and we also have a quick video if you’re more of an audio/visual learner.
Now, let’s get on with the good part about writing user scenario scripts for your e-commerce checkout process.
2. Payment Testing Best Practices
You’re selling something, and there’s a good chance you use a payment provider to collect information (credit card info, etc.)
These providers typically provide some kind of QA/test environment with a number of possible payment methods, but it’s important to know that many payment providers don’t want you to include their service in your load tests.
Simply put, payment providers don’t want you running a load test with hundreds of thousands of Virtual Users and crashing their product. (Seems like a reasonable request.)
If you think it’s absolutely imperative to test the actual payment-transaction process in your load test, you should probably notify your payment provider first. Some processors are even nice enough to provide sample payment data to use for tests or allow you to place your account in testing mode.
In some cases, creating simulated requests against your payment provider’s system could violate your terms of service, as these companies are trying to protect themselves from performance issues.
So, take a look at your TOS or just send an email to your payment provider’s support team to make sure you’re in the clear.
3. Testing Email or SMS
Customers usually expect to receive some kind of email confirmation after making a purchase, and many of our customers have asked about including that in their load tests.
Our advice is to set up a catch-all mailbox to receive all those test emails.
A load test can generate thousands of emails (depending on the number of Virtual Users you’re testing), and sending all of those in a short time frame without configuring your email settings could cause you to be blacklisted or flagged as spam by an email provider.
Also, if your purchase confirmation is sent via SMS, then you’ll probably want to just disable that when running your load tests. Depending on how you pay for that, sending thousands of test messages could be a waste of money.
4. Handling Inventory
This is a big one for our e-commerce friends working with a supply chain.
If you’re including real items in your test, Virtual Users running in your test scenario can cycle through the scenario several times, which means they’ll just keep “ordering” an item until the test finishes.
Depending on the level of automation you’re using in your logistics, this could lead to your system automatically re-ordering items you don’t need.
So, make sure you’re either testing something with plenty of inventory, or configure some items specifically for your load tests that won’t “run out” of inventory or trigger an auto-reorder.
Most of our e-commerce users will create a few fake SKUs to be included only in load tests to accomplish this.
5. Testing Backend Integrations (Logistics, etc.)
In the same vein as inventory, you’re probably using other integrations — invoicing, logistics providers, printing, CRM, financials, etc. — that could be affected by hundreds or thousands of Virtual Users accessing your website and “ordering” things.
Keep all of those in mind and consider using only test accounts or test data when creating your user scenarios and tests. It could save you from a massive (and unnecessary) headache down the line.
6. Creating Sufficient Test data
Load tests consume data very quickly. You definitely want to make sure your Data Stores have plenty of information for the Virtual Users in your load test.
Let’s outline a quick breakdown to show what we mean. Let’s say you’re running a test with this configuration:
- Ramping up to 5,000 concurrent Virtual Users
- Test duration of four minutes
- Testing process of user logging in with a username and password, searching the product listing, adding an item to a cart and checking out
- Various user scenarios take around two minutes each to complete
*Quick note: Four minutes is a super short duration for a test, but it made the math simple to understand for the sake of the example, so let’s just roll with it*
So, if you want every scenario to be executed with a unique username and password, you’re going to need to include about 10,000 lines in the CSV file you convert to a Data Store.
Based on the scenario above, each user scenario will be completed about twice within the test duration, so that’s why you’ll need about 10,000 unique rows in your Data Store.
This also highlights an important lesson for anyone who’s performance testing: Load tests can run through parameterized data quickly depending on the length of your user scenarios compared to the duration of your tests, so make sure you have enough lines in your CSV files to handle that scale.
7. Using Tokens: CSRF Tokens, VIEWSTATE, etc.
This can be an unforeseen wrinkle for some engineers who haven’t been working specifically in QA and performance testing throughout their career.
For many of our users who utilize the Load Impact User Scenario Recorder Chrome extension to easily create testing scripts, the hiccup starts when our scenario recorder captures their specific CSRF token.
In order to properly test unique CSRF tokens for each of your Virtual Users in your test, you’ll need to extract the tokens, save them to variables, concatenating them it into future relevant requests.
There’s a bit of scripting involved in that, but here’s an easy reference guide (with a theoretical example!) to help you load test a website with CSRF tokens or VIEWSTATE.
Those are just a few recommendations. There are hundreds of other tips, tricks, FAQs and general life lessons about load testing in our Knowledge Base.