Design E-commerce app to display Top Selling Products, daily sales report and seller’s sale Performance
What we have to Cover
1. There is a running e-commerce website with more than million orders coming per day and more than billion search and browse request on various products. You have to display the top selling products of the category that user is currently searching for. There should be a new endpoint to provide top selling product for specific category.
2. The new point should able to support billion+ requests per day.
3. Based on the daily orders generated, a sales report should get generated in specified format and get inserted in any storage from where it can be picked, parsed and analyzed by the various concerned teams.
4. Extend the existing system to facilitate/integrate with product sellers and to periodically tell them their product sales rank.
5. System should be built for hourly update of top product result.
6. Generated orders data could be batch (hourly, 15 minutes, 30 minutes, etc.) or real time. Should support both type of ingest.
7. There shouldn’t be any data loss and system should be scalable enough to handle higher load in peak hours or sale hours.
8. Seller should be notified for new reports via email, SMS and App based notification. Notification can be sent in batches via file ingest or real time.
High level Architecture
Design Decisions
1. For ingestion of Top Selling product data and Seller product sales rank data, Redis will be used as its a cached based storage and works on hashing mechanism with key-value pair. For Top Selling Product Cache Category Id will be the key and related data will be value. For schema details, please refer to schema section. To Deploy Redis Farm will be used as high TPS is required and Deployment mode will be multiple master as availability is priority over consistency to support >200 ms time response under high TPS load.
2. For mailing, Amazon SES is used as SMTP server but hosted server can also be used. Notification will be delivered by centralized notification app primarily built on AWS manages services ( SQS, SNS, Lambda). For creating the notification channel, refer to centralized notification system.
3. Functional assumption is been made that we need to ingest only latest upcoming data, so data in Redis will be replaced for every key(category Id or Seller Id) every hour. If we need to carry previous record as well, then job pipeline will have to fetch existing data from Redis and bake that data adding it to the latest file records.
4. To avoid data loss we have placed Messaging and streaming system for components to inter-communicate.
5. Apache Spark/Apache Beam will be used as data transformation pipeline. Files will be ingested into these data transformation pipelines for parsing, comparing and decision making. Data will be transformed using two separate pipelines (one for top-selling products and one for seller performance).
API(s), Schema and Schema Design
