Don’t say goodbye to your loved ones: how not to lose customers in the “high season”

Guess a number between 40 and 60. This is about the percentage of users who won’t wait more than 2–3 seconds for a website or mobile app to load. Think about it. Just three seconds of waiting and they’ll head off to your competitors, and you’ll lose potential revenue.
Yuri Kovalev, Managing Partner at Performance Lab, explains how not to lose customers in your “high season”.
State of affairs
December revenues for Russian online stores is typically 1.5–3 times higher than in November. Most often, December brings in the biggest revenues for the year. Beginning with the end of November, crowds of shoppers head out to make their purchases. They go to online stores and spend a lot of money there. A distinctive feature of this period is that buyers are looking for a way to spend money quickly.
Statistics say that 75% of customers will go to another online store if they encounter a slow website or mobile app. Most sources agree that buyers are becoming less patient every year. As the online shopping segment expands, customers’ patience is rapidly declining.
Following the US tradition, in recent years Russian retailers have created a Black Friday event, a day with the biggest discounts of the year to attract a large number of buyers. Usually Black Friday takes place at the end of November, which marks the beginning of the high season lasting until the end of December.
Some statistics:
· Consumers spent $453.46 billion on the web for retail purchases in 2017, a 16.0 precent increase compared with $390.99 billion in 2016. That’s the highest growth rate since 2011, when online sales grew 17.5 precent over 2010.
· Mobile traffic reached 63 precent of all traffic in the US in 2017.
·In 2016, the number of smartphones sold to consumers stood at around 1,5 billion units, a significant increase from the 680 million units sold in 2012.
·Mobile sales now accounts for six precent of total retail sales and nearly 60 precent of e-commerce, following a yearly rise of 40 precent in 2017
· Nearly 62 precent of smartphone users have made a purchase online using their mobile device in the last six months.
· 79 precent of web shoppers who have trouble with web site performance say they won’t return to the site to buy again and around 44 precent of them would tell a friend if they had a poor experience shopping online.
Considering the figures above, we can say that load testing has never been more important for online and mobile retailers.
How can you reduce the likelihood that your website or app will fail during high season? How can you ensure it will meet the required performance standards? Be sure to do performance testing. Start testing as early as possible. Test constantly.
1. Create a test version of your website
To start, make sure that you have a suitable test environment. Testing the performance of a live production website is dangerous.
1. First, because doing so can cause it to fail, making active users suffer.
2. Second, because during testing new orders will be created, and test payments and other operations will be performed, which will lead to undesirable consequences. For example, your real customers will start receiving email notifications about purchases they didn’t make.
Ideally, a test environment for stress testing before the high season should be a complete copy of your real production infrastructure. This means that the test equipment’s capacity must be just like your “battle-ready” system. And the configuration of servers and applications should also be the same as much as possible.
2. Analyze your store statistics
Once the question of the test environment is resolved, bring in analysts to accurately simulate the expected user load. Load testing won’t provide any benefits if thousands of users perform the same operations, e.g. signing in to their accounts.
Testing must be realistic, and this requires the correct load profile. To create it, analyze the statistics from the online store’s database or from a service such as Google Analytics.
A good result will be a load profile in which user groups performing specific operations (based on the statistical data) are assigned percentages. For example, 20% of users navigate from the main page to the product listing, 5% go to their account settings, 10% use the search function, and so on. Such a load profile will let you simulate realistic user behavior regardless of the number of users.
3. Create virtual users
After preparing the load profile, you need to create virtual users for each group you defined in the previous step. This is done using special load testing tools such as JMeter. This requires programming skills and tool configuration, so it’s best to have experienced engineers do this work.
Why do we test on virtual users and not on real ones? Well, can you imagine how many real users you would need, and then to have them all simultaneously do the specified test scenarios?
4. Test using different loads: will the website survive if 100 customers arrive at the same time? 100,000? 1,000,000?
Now you can get down to testing. Test using different loads: average, peak, stress, dramatically increasing, long (to detect memory leaks). Your system can operate normally with a normal load, but what happens if the load doubles or triples? Or unexpectedly spikes?
Testing for memory leaks should be performed over an extended period (12–48 hours). An application may deal well with a high load at first, but then malfunction due to a memory leak.
During testing, you need to watch metrics such as response time (the time required to perform specific operations, load specific pages), CPU load across all servers, memory usage, and disk subsystem utilization. As a rule, given a certain increase in the load, response times begin to increase. For example, the website’s home page may take 11 seconds to load instead of 3.
If the load is increased further, then the website stops responding at some point. By analyzing these indicators, you’ll learn how well and quickly your website works with various numbers of users.
5. Don’t forget about problems on the client side: will an IE user be able to open your store in her favorite browser or will she have to install Chrome for this?
Test the performance of the client portion just as carefully as the back-end. The fact is that slow loading of webpages may be caused not only by an overloaded server, but also by the behavior of the webpages in the client’s browser.
For example, slow JavaScript can cause a several-fold slow down in page loading. Is your app optimized for different browsers and different devices? Model the bandwidth of different channels during tests: how does your app work over Wi-Fi, 3G, and 4G? Users don’t care why the app failed. They expect the program to be fast in all conditions.
If you have an in-house development team, have them analyze app behavior during load tests. This will greatly speed up the search for and elimination of bottlenecks in your system.
6. Track your website’s performance in real time: learn how to anticipate problems before they occur
And finally, in addition to performing load testing, use monitoring systems to be sure that your system’s basic performance metrics “in battle” (i.e. your real online store, not your test store) do not fall below certain thresholds. Learn to recognize patterns in the monitoring data and use it to anticipate imminent problems while you have time to take action.
