Distributed Systems — System Design Concept

Introduction

Nowadays, almost every large-scale application is based on distributed computing. So, It is crucial to have knowledge of distributed systems as they are instrumental in designing fault-tolerant, highly scalable, and low-latency services.

This blog will introduce you to the fundamentals of Distributed Systems, their functioning, and their real-world applications, using some exciting examples. So, let us dive in and learn how to deal with trade-offs while designing our system.

What is a Distributed System?

A Distributed System is a collection of many independent machines that work together by coordinating and communicating with each other to achieve a common goal. These systems operate concurrently, offer high scalability, and fail independently without affecting the application’s performance.

Distributed Systems work in a fashion that makes them appear as a single entity to the end user. They allow for resource sharing, concurrency, fault tolerance, scalability, and transparency.

Why Distributed System?

Distributed Systems are composed of machines capable of connecting over a network, having local memory, and communicating messages. Hence, it is evident that such systems consist of a vast amount of complexity. So the common question is — why are we using Distributed Systems if they are so hard to manage, deploy, and debug due to their extremely complex behaviour?

The most significant advantage of adopting Distributed Computing in almost every software is its ability to provide highly scalable services. Distributed Systems allow us to add as many servers as needed, offering Horizontal Scaling.

• Traditional systems generally scale using Vertical Scaling, meaning by increasing the single machine’s complexity by upgrading it multiple times. Such Vertical scalable services are not compatible when the service is operating on a vast scale because they are expensive and more prone to single points of failure.
• However, horizontal scaling allows us to scale indefinitely without any worry. If there is any performance degradation, we only need to add more machines, making the service extremely fast without much overhead cost compared to Vertical Scaling.

Along with scalability, Distributed Systems offer low latency and fault-tolerant services. Most highly scalable services are read-heavy, which may decrease the system’s performance. To deal with this, distributed computing allows servers to be replicated, providing highly available services and ensuring fault-tolerant behaviour.

However, while database replication can effectively address low-performance issues, there is a specific limit to its convenience. With distributed systems, there is always a trade-off between complexity and performance.

To further increase performance, distributed systems offer another way to scale the service by sharding the databases. With sharding, the main server can be split into many smaller servers called Shards, allowing for the distribution of the load and achieving highly available and low latency services.

How Distributed System Works?

Distributed Systems heavily rely on network calls and communication from all the individual components. A well-established network is the utmost requirement for the Distributed System to connect with each component and offer highly scalable services.

The messages need to be communicated reliably. There should be a mechanism for detecting any failed node so that if the node fails, the System should synchronize properly without affecting other functionalities.

The distributed systems fall into any one of the four architectural modes:

1. Client-Server Architecture: Most of the traditional architecture falls under this category. There is a server to which all the requests are made from the clients. Resource sharing is one of the best examples of the Client-Server model.
2. Three-Tier Architecture: In such architecture, a central server manages all the requests and services and acts as a middle layer between the communication. The middle tier accepts the request, does some pre-processing, and forwards it to the server for further processing.
3. Multi-Tier Architecture: Such architectures are used when an application needs to forward requests to various network services. Here the application servers interact both with the presentation tiers and data tiers.
4. Peer-to-Peer Architecture: There are no centralized machines required in this architecture. Each entity behaves as an independent server and performs their roles. Responsibilities are distributed among various servers called peers, and they cooperate to achieve a common goal.

Examples which are heavily relied on Distributed Systems are:

1. Telecommunication Networks
2. Parallel Processing
3. Real-Time Distributed Services
4. Distributed Databases and Caches
5. Distributed File System
6. Stream Processing Services

Advantages of Distributed System

Nowadays, systems require high availability, consistency, scalability, and fault tolerance. Hence, the vast majority of applications rely on distributed systems. The key benefits of using distributed systems are:

• Reliability: Distributed systems remain available most of the time, irrespective of the failure of any particular server in the system. If one server fails, the service remains operational.
• Scalability: Distributed systems offer extensive scalability. The advantage of adding a large number of servers allows such systems to achieve horizontal scalability.
• Low Latency: Distributed systems offer high-speed service because of server replication and their location close to users, hence reducing query time.

Disadvantages of Distributed System

It is essential to know the various challenges that one may encounter while using any system. This will help in dealing with trade-offs. Distributed Systems offer high performance and scalability, but they are highly complex. The shortcomings of Distributed Systems are:

• Complexity: Although a large number of machines can be used to make the system scalable, it increases the service’s complexity. There will be more messages, more network calls, and more machines, and hence it becomes very tedious to manage such systems.
• Network Failure: Distributed Systems heavily rely on network calls for communication and the transfer of information or data. In case of network failure, message mismatch or incorrect ordering of segments can lead to communication failure and eventually deteriorate the application’s overall performance.
• Consistency: Due to its highly complex nature, it becomes challenging to synchronize the application states and manage the data integrity in the service.
• Management: Many functions, such as load balancing, monitoring, increased intelligence, logging, etc., need to be added to prevent failures of the distributed systems.

Conclusion

Distributed Systems are a necessity of modern applications that are heavily relied on distributed computing. Such systems can support many requests and compute jobs compared to a single standard system by spreading workloads and requests.

Although there are some trade-offs and challenges, Distributed Systems can transform the world with its services and applications and as a result of which almost every application have Distributed System as their one of the major component.

For more content, explore our free System Design Course and System Design Blogs. If you have any queries or feedback, please write us at contact@enjoyalgorithms.com. Enjoy learning, Enjoy algorithms!