System Design : Shots

A gist of System design basics, explained in as few lines as possible!

1. Caching Techniques:

• Cache Aside: In this technique, data is fetched from the main storage on demand. If the data is not found in the cache, it’s fetched from the main storage and then stored in the cache for future access, reducing subsequent retrieval time. To avoid stale data problem, one can adapt TTL, and cache invalidation/update upon a DB write.
• Write Back: Data is written to the cache and then asynchronously updated in the main storage. This optimizes write performance by reducing the number of write operations to the main storage, as multiple writes can be consolidated and performed asynchronously. The main tradeoff is on the storage requirement since we do every write to cache. TTL can be used to mitigate this problem, but use this strategy when you have read heavy systems, because write is a costly operation.
• Write around Cache: In this approach, data is written directly to the main storage, bypassing the cache entirely. It’s useful for large, infrequently accessed data that doesn’t benefit from caching, avoiding cache pollution and reducing cache churn.

2. Caching Challenges:

• Stale Data: Caches can contain outdated information if data changes in the main storage and the cache isn’t updated accordingly. Techniques like cache invalidation or expiration help mitigate this issue.
• Cache Churn: High turnover of data in the cache due to frequent updates or eviction can lead to increased cache misses and reduced effectiveness. Optimizing cache eviction policies and cache size can alleviate this problem.
• Cache Coherence in Distributed Caching: Ensuring consistency across distributed cache nodes is challenging due to network latency and node failures. Techniques like cache invalidation, distributed locking, and consistency protocols help maintain cache coherence.

3. Rate Limiting Algorithms:

Rate limiting algorithms control the rate of incoming requests or data flow to prevent overload and maintain system stability. Common algorithms include token bucket, leaky bucket, and fixed window. Token Bucket:

• Token Bucket is a rate limiting algorithm where tokens are periodically added to a bucket at a predetermined rate. Each request consumes tokens, and if the bucket is empty, the request is delayed or dropped. Use cases include network traffic shaping, API rate limiting, and controlling access to shared resources like file systems.
• Leaky Bucket is a rate limiting algorithm where requests are processed at a constant rate, akin to water leaking from a bucket. Incoming requests are stored in the bucket, and if it overflows, excess requests are either discarded or delayed. It’s used in network traffic policing, preventing bursts of traffic from overwhelming systems, and smoothing out traffic flows.
• Fixed Window is a rate limiting algorithm that calculates the number of requests made within fixed time intervals. Requests exceeding the defined limit are rejected or delayed until the next time window. It’s applied in API rate limiting, preventing spikes in traffic, and ensuring fair resource utilization across users or applications.

4. Kafka Discussions:

• Ordering: Kafka maintains message ordering within each partition, ensuring that messages are consumed in the same order they were produced.
• Scaling Kafka: Kafka achieves horizontal scalability by partitioning data across multiple brokers, allowing for distributed storage and processing of messages.
• Message Size limit: Kafka configuration limits the size of messages that it’s allowed to send. By default, this limit is 1MB. In case of large files, it is a goot idea to upload them to S3 and send the url in kafka message.
• Dead Letter Queue: Use this as a topic to implement retries for failed message processing, where a retry consumer can process messages in this topic.

5. Bloom Filter:

• Bloom Filter is a probabilistic data structure used to test whether an element is a member of a set. It efficiently represents a large set with a compact data structure by using multiple hash functions and bit arrays.
• While it can efficiently determine the presence of an element in a set, it may occasionally produce false positives, indicating that an element is in the set when it is not. However, it never produces false negatives.
• Bloom Filters find applications in caching, spell checkers, network routers, and duplicate elimination tasks where false positives can be tolerated.

6. Consistent Hashing:

• Consistent Hashing is a hashing technique used to distribute data across a dynamically changing set of nodes in a distributed system.
• It minimizes the need for data migration when nodes are added or removed from the system by ensuring that only a fraction of the data needs to be remapped.
• Consistent hashing achieves this by mapping both data and nodes onto a common hash space, ensuring that similar hash values are assigned to nearby nodes, thus reducing the impact of node additions or removals on the overall system.
• It is commonly used in distributed caching, distributed databases, and content delivery networks (CDNs) to evenly distribute load and ensure fault tolerance.

7. Load Balancing Algorithms:

Round-robin: This algorithm distributes incoming requests or traffic equally among a set of servers in a sequential manner. Each new request is forwarded to the next server in the list, looping back to the beginning once all servers have been utilized. Use cases : Web server farms, Load Balancing in DNS, Email server clusters

Least connections: With this algorithm, incoming requests are directed to the server with the fewest active connections at the time of request.

Application servers handling long-lived connections, Streaming media servers are typical use cases.

IP hash: In this approach, the IP address of the client making the request is used to determine which server should handle the request. The hash function generates a unique identifier based on the client’s IP address, ensuring that requests from the same client are consistently routed to the same server. This helps maintain session persistence and is commonly used in session-based applications or when stateful connections need to be maintained. Geo location based routing is another use case.

Weighted round-robin: Similar to round-robin, but with the added capability of assigning different weights to servers based on their capacity or capability. Servers with higher weights receive a proportionally higher number of requests compared to servers with lower weights. Commonly used in Multi-tiered architectures, and Disaster recovery

8. Database Use Cases:

• Mysql: Suitable for transactional applications requiring ACID compliance, relational data modeling, and structured queries.
• Postgres: Offers advanced features like JSONB support, full-text search, and extensibility, making it suitable for complex data types and applications requiring strong consistency and reliability.
• MongoDB: A NoSQL database ideal for flexible, document-oriented data models, scalability, and horizontal scaling. It’s well-suited for applications with evolving schemas and high write throughput requirements.
• Cassandra: Designed for high availability, scalability, and distributed data storage. It’s suitable for applications requiring linear scalability and eventual consistency, such as time-series data, IoT, and real-time analytics.
• DynamoDB/CouchDB: Highly scalable NoSQL databases optimized for distributed environments, offering features like automatic sharding, eventual consistency, and built-in replication. They are suitable for applications requiring high availability, low latency, and seamless scalability.