How do we show discounted prices on product listing pages?
Hello everyone,
As you know, there are millions of products in the Hepsiburada system, and these products are searched and listed by customers through the search bar. Customers add products that meet their criteria to their carts and place orders.
When considering customer experience, discounts on products and showing the final discounted price applied to the product (on search pages) before the user goes to the cart can be a significant benefit for both the user and the application. Therefore, as the promo team, we strive to present the discounted price of the product to the customer in many places on the before sales side.
One of these important pages is the product listing screens.
Here, our goal is to provide a better experience for the customer by directly showing the discounted price of the product without the customer having to add the product to the cart or open the product detail page.
In the visual above, the texts you see as “Sepette 99,45 TL”, “Sepette 212,71 TL”, and “Sepette 88,40 TL” are fed in real-time from the RESTful service developed by our team. On the product listing pages, there are challenges such as high throughput of requests to the service due to real-time campaign queries specific to each product, as well as the need for our response time to be low.
So, how did we handle these challenges?
- Network:
Here, we deployed the API prepared with the aim of eliminating HTTP traffic between clusters because the cluster containing the services feeding the product listing pages and the cluster containing the services of the promo team were different. This deployment allowed us to serve the listing screens’ API directly within the cluster where they are needed. This approach minimizes the time lost in the network layer.
- Data Storage:
As you might anticipate, under high load, we couldn’t fetch the active campaign list from the database every time. We needed a well-structured caching mechanism that wouldn’t impact memory.
To access the data quickly, we opted for in-memory caching among various cache strategies. However, as with any choice, it came with its own set of challenges.
1- How long should our cache duration be?
2- Minimizing the time lost during the deserialization process of the large data obtained from the cache under high load.
Here, we decided to implement the Singleton pattern. During the bootstrap phase, we filled the singleton object, which is the campaign list within the service, by fetching active campaigns from the campaign service.
We couldn’t afford to go to the campaign service every time to fetch active campaigns due to performance concerns. As active campaigns could change frequently, we needed to keep our singleton list constantly updated.
To avoid the cost of bulk fetching all active campaigns periodically, we utilized Redis’ pub/sub capability. When a campaign was created, updated, or removed from the campaign panel, an event was triggered, allowing us to update our singleton list. This approach gave us the flexibility to extend our cache duration up to 24 hours because we could refresh the campaigns instantly.
We also set up a scheduled job to fetch all active campaigns once every 24 hours and migrate them to the singleton object, completing the data storage part of the process.
You can see the API architecture drawn by Mahmut Özben below:
As you can see above, when the API’s pods are first deployed, they fetch active campaigns from the campaign-api. Afterward, when there are changes in campaigns from the campaign panel, the event related to the campaign is triggered using Redis pub/sub, updating the singleton object in each pod of the application.
- Calculating Campaigns:
Request model four our API:
The request payload sent to our API, can contain multiple product details. Information such as stock code, category, brand, and product group is already prepared and transmitted to us by the team using the service. This allows us to perform all our evaluations using this information about the product without the need to access an additional product service.
Now, let’s delve into what campaign evaluation means in a bit more detail.
Our campaigns can be defined based on product attributes such as brand, category, stock code, and product group. For example, a 10% discount on baby products. By comparing the conditions on the campaigns with the product information received in the request payload, we determine which campaigns can be applied to the products. We then find the relevant campaigns in order of priority from these filtered campaigns, calculate the final price for the product, and return it in the response.
After freeing ourselves from the costs of database and HTTP traffic, the challenge we needed to address was performing this evaluation process as quickly as possible. To achieve this, we initially considered evaluating the incoming list of products separately on a per-product basis using individual threads. Since the products didn’t have relationships with each other, there was no need for the process to be synchronous.
The evaluation process could be done in parallel without waiting for one product to finish before evaluating the next. However, after implementing and testing this process, the response time didn’t turn out as we had envisioned.
As seen above, we observed that the response time increased to 260 ms at a throughput of 20k-25k requests per minute. From this, we drew two conclusions:
1- The number of incoming requests was overwhelming, causing a significant increase in CPU usage. As a result, our cluster experienced congestion.
2- The campaign data was too extensive to iterate through within the list and check individual conditions one by one.
So, how did we tackle these new challenges we encountered?
Our solution to the problem of the campaign list being too large for the evaluation process was:
The solution that came to our minds for this problem was to partition our campaign list. If we divide our data based on campaign conditions (such as merchantId, categoryId, sku, productTag, brand, etc.) and convert this data from a list to a dictionary, we could use the campaign’s condition parameter as the key for the dictionary. Then, with the product sent in the request, we could directly perform a key search in the dictionary. This way, we wouldn’t have to iterate through the list item by item.
Let me elaborate on this with visuals, as it might be a bit complex to explain verbally.
I’ve already shared an example of the request we receive. It’s important to note that conditions like category, sku, and brand are sent in the request along with the product.
Let’s consider a campaign applicable to a seller called Seller X. When we refactor our data in the cache, using Seller X’s ID as the key and this campaign as the value, what do we gain?
Instead of searching for a campaign that could potentially match the product’s attributes one by one within the campaign list, we directly retrieve the campaign from the dictionary using the product’s merchantId. This not only saves us from the cost of iterating through a large campaign dataset but also provides us with additional speed due to key search in the dictionary.
But are all campaigns in our system defined only for sellers? Of course not.
To handle this, we decided to maintain our data in separate dictionaries. For example, a campaign defined for a category is stored in a dictionary named `categoryCampaignDictionary`, while a campaign defined for a brand is stored in a dictionary named `brandCampaignDictionary`. We evaluated these dictionaries in parallel threads. This approach gave us a total of 7 threads, reducing the thread count and solving our first problem.
We populate our dictionaries as described above. For each `merchantId` in the conditions of the respective campaign, we assign separate keys. Since there could be multiple campaigns defined for the same merchant, we define our values as arrays and link them with `else if` statements to prevent duplicate campaign information in multiple dictionaries.
I can sense you asking, ‘Aren’t there any race conditions with 7 threads running?’ Yes, indeed, this is another challenge we had to handle.
We prevented the race condition as follows:
We didn’t want to use locks as they could slow us down even at the millisecond level. To prevent this, we ensured each thread completed its operations and prepared separate response models.
go func() {
// preparing first temporary response dictionary
}()
go func() {
// preparing second temporary response dictionary
}()For instance, as you can see above, two threads are running, each filling different response models. The response models you see here are our temporary models represented as dictionaries. We used the product’s listingId as the key and assigned the respective campaign as the value. This approach created a key-value working environment for us, providing an additional speed boost.
After the threads finished their work, we had 7 dictionaries. We performed key searches on these dictionaries based on the products in the request, collected the campaign data that matched the products, and presented the most beneficial one to the user.
Benefits of our approach in overcoming this challenge:
We avoided race conditions by storing each thread’s response in different temporary response models.
We didn’t experience any delays, even in milliseconds, because we didn’t use locks.
By using dictionaries for our temporary response models and performing key searches based on the product’s listingId to find the most suitable campaign for the product in our final response model, we gained speed.
After our recent enhancements, the response times of our service are as follows:
As you can see, we have an average response time of around 4 ms with a throughput in the range of 150k requests per minute. If you recall, when the service was first developed, it used to operate at around 260 ms under a load of 20k-25k requests.
Indeed, we had to challenge many aspects from the beginning to the end of this project, and as we solved one, another one emerged before us. We have learned a lot while overcoming these challenges. I want to thank you for reading my writing where I share what I have learned. I hope the challenges we faced and the lessons we learned will also be useful for you.
Finally, I want to express my endless gratitude to Mahmut Özben, Çağlar Demir, Anıl Güzel, Tugay Kılavuzoğlu, Alpcan Gazan, and Gökhan Tüylü for their efforts in this project. Well done, team! :)
