System Design Fundamentals — part 2
Read part-1 here: https://medium.com/@prudhvir3ddy/system-design-fundamentals-part-1-2ef1865e14a5
In the first part, we saw some vocabulary and this part continues the same, we will see all the pieces available to us before going to connect them or the connection problems.
Load Balancing:
When there are a million requests coming from end-users if we have a single server it will be too much burden on it, so as discussed in the previous blog we need to do horizontal scaling and add more servers.
the load balancer job is to make sure it distributes the incoming requests to one of the servers in the pool of the servers.
on what basis a load balancer will send the request to a particular server in the server pool? there are different algorithms for that. these are some of them learning a little will make sense of what it is doing actually
- least connection method — selects the server that has the least connections
- least response time method — selects the server that has the lowest average response time
3. round-robin method — select the server on basis of this formula = ( request % (total_number_of_server) ) assume we name server as 0,1,2,3…n servers
4. I.P hash — selects the server
in order for a specific server to be available in the server pool two things are required:
- a server should respond to the requests
- a server should pass health checks — system health checks will make sure that the server is not infected or some functionalities in the server is working as expected
when server health check is failed load balancer will not send requests to it unless the health check passes again
also, we have only one load balancer here. is one enough? yes, one is enough as its job is only to route traffic with small analysis.
quick tip:
single point of failure concept is “what happens when our server got shut down or destroyed for some reason. it can be due to weather conditions, electrical accidents, or human error. ”
so in our design, we always have to make sure there are no single point of failures
but in above we got only one single load balancer, what happens if that fails?
add another redundant load balancer which checks for health of the first load balancer and when it fails it switches up and takes place of primary load balancer making it the secondary — imagine coding this🤪. no, let’s just stick to standard concepts for now.
Caching
caching is such an amazing concept that I started using recently in my applications.
caching is a way to get data fastly that is frequently accessed and storage of cache is usually less than original
let’s take an example: I made a project which fetches news from different sources so the first time you open the app it’s gonna take some time to fetch news from the server. now, for checking some message you went to a different app and if you come back you expect news to be there and not show loading again and again which is a wastage of internet in user mobile and making the server overload with requests so in the above case you need a cache mechanism so that user can see the news there and you need a place where user can get the news fastly and it’s definitely not in any server because that requires internet. so, store them in the user local mobile database where querying the data is fast.
this is just one use case of caching depending on the situation you need to change the cache mechanism and remember caching can be applied to every level of the system
choosing the cache mechanism and implementing it is not much work, straightly imperative but remember you need to invalidate the cache when new data is available and don’t show the old data
Content Delivery Network ( CDNs )
CDNs are good for story static content. suppose if your site contains just some static data and it is often requested then you should go for CDN. again, whenever we talk about system design we should assume the worst can happen like a million users opening the site and it’s not a good idea that our server should serve the same content again ( unnecessary processing of requests, we know what to send as a response)
Connection Responses From Server:
Http Polling
HTTP polling is a concept where you continuously send the request to the server in some interval like 5 seconds and when the server has the data it sends the response when there is no data we receive the empty response.
Http Long Polling
HTTP long polling is the client sends the request to the server and there will be a connection open till the server responds back with a non-empty response. once the client receives the response then again the client sends the request to the server, so there is always a waiting request open at the server.
Web Sockets
web socket is more of a two-way connection, once the connection is established over TCP, the server can send data any time to the client and the client can send data to the server at any time.
just like the Whatsapp whenever you are connected to the internet a socket is created and message exchange is real-time without any delay, whenever you disconnect the internet the connection closes.
Server-Sent Events
here the client opens a connection with the server once and the server will send updates to the user but the client cannot send the requests over this connection and just receive the updates.
for some news kinda applications, this suits well as users just need to grab the data and not send anything.
Proxy servers
simply masking our existence when requesting resources from the server
there’s also reverse proxy server which accepts requests from clients but processing is done by one of the servers in a pool of servers available and knowledge is only available with the reverse proxy server but as far as client knowledge the request is handled by a reverse proxy server that we requested
Thanks for reading this blog. hope you had learned some vocabulary of system design from this blog. if you loved it then give some 👏 s to the blog. please drop a comment incase things are not clear at any place.