7 Steps to winning Black Friday
Black Friday is unlike any other working day for an eCommerce company. This year we were very well prepared and thankfully the day went without too many glitches and we beat all the records.
This article will provide a more high level overview and covers some more organizational topics…for more details on the tech preparation, please read the more in depth article by our CTO Jan-André le Roux below: https://medium.com/tech-travelstart/preparing-for-a-black-friday-1cdf1754f8f2.
- Scalability improvements
Our CTO Jan-André le Roux lead the development team into making quite a few changes that that makes our site even more scalable. These changes involved things like optimizing the use of the Cloud storage & Content Delivery Network (CDN) for delivery of static resources as well as more caching of internal and static data to reduce the number of internal API calls and database connections.
2. Provisioning and Load testing
In preparation for the big day provisioned quite a lot of additional servers to deal with the load which sometimes can get up to 25x the normal traffic. We also optimized the Flight Cache rules and internal jobs run times based on the very specific requirements for Black Friday. To know that all the optimizations worked, we did quite a bit of Load Testing the week before. These tests actually highlighted some infrastructure bottlenecks that we managed to solve in time. To make sure our Operations team would not run out of juice either, we seriously upgraded our Internet bandwidth for the week and tweaked our Voice platform.
3. Monitoring & Alerting
During the day, the whole team needs to be able to monitor the system so we implemented additional dashboards, each with their own purpose. On the day the whole team sat together in one room where appropriately the Star Wars movie theme was played. Just in case we would miss something, the alerts were also tuned to make sure we would be notified as soon as possible. Parts of the system we closely track are supplier response times, hardware behavior and the response times of critical parts of the system (e.g. search and book).
4. Risk Mitigation
We know that on the day our external providers (e.g. payment gateways or airlines) suffer as well as we do and it’s not uncommon on the day to have issues that are not under our control. We identified scenarios for most of the risks and identified what to do to to mitigate the risk and/or how to respond to the risk should this occur. In most scenarios and if possible, we still accept the customer’s order while trying to process the reservation and/or payment afterwards. Thankfully only 2 of those scenarios actually played out and we knew exactly what to do.
5. Managing the load
Thankfully our marketing department understands that managing the load will help to keep the system stable and as such, the different deals were made available to our customers throughout the day and our messaging and notifications were sent throughout the day (as well as the following days). With each communication we noticed a significant increase in traffic but it never got out of control.
6. Business Rules
Our system needs to be configured to make sure the supplier deals show on the website and that the right discounts are applied. For this we rely a lot on the airlines to make the deals available to us which they obviously do as short as possible before the actual time. This implies that our Technical Supply team need to be up at all hours and that they work closely with both Marketing as well as Supply to test the changes carefully and ensure that customers can successfully book.
7. Communication channels
Slack turned out to be a major asset for us. Last year we were still running around like headless chickens and sending mails which 3 minutes later were old news. This year however all information was flowing very well between our IT team, the Call Center, Marketing and Product through different Slack Channels. Each channels had a different purpose so we didn’t bore the call centers or marketing much with the IT specific communications.
Thankfully Black Friday turned out to be a great day for Travelstart, not only because IT had done it’s job but because the whole company was working closely together on delivering a fantastic day.
