Instagram System Design

Understanding the System

What is Instagram?

Instagram is a free social-media application for sharing photos and videos with your followers. It has an array of features, from short-form videos to live streaming and private messaging.
I’m sure each of us would have definitely used this application atleast once. But, have you ever wondered how does this application is internally designed? Also, what are the key considerations to be taken into account while designing applications like Instagram.

Requirements

Functional

1. User should be able to upload photos and view uploaded photos
2. User should be able to like and comment on the posts
3. User should be able to search photos/videos based on title
4. User should be able to follow other users
5. User should be able to view the top feeds from the users they follow

Out Of Scope

Sending or Receiving Messages to other users

Non-Functional Requirements

1. The System should be highly available and should have low latency with user feed generation. Here, we will be maintaining eventual consistency, wherein it should be acceptable, if users are seeing an image after a few milliseconds.
2. The System should be highly reliable ensuring that the photos/videos are not lost.
3. The System should be highly scalable for read-heavy workloads.
4. The System should be optimized for popular posts by celebrity users. Scenarios like 1 celebrity picture upload should be viewed by 1M users.

Capacity Estimation

Let’s look at the capacity to get Traffic and Storage estimates, assuming that there are 500M registered users and out of which 100M are daily users.

Traffic Estimates

Assume Number of daily users - 100M
1% are Active Users (users uploading photos/videos) - 1M
Assume 5M photos are posted daily - ~57 uploads/Second

Storage Estimates

Considering the below storage estimates, we would need 3 copies of the storage. Along with that storage for metadata would also be required.

Average size of a photo - 200 Kb
Storage Size per day - 200 Kb * 5M= ~1TB/day
Storage Size per year - 1TB * 365 = ~350 TB/year

Average size of video - 50MB
Storage for videos per day for 1M users - 1M  x 50MB = ~50 TB/day.

High Level Design

Instagram being a complex system has the following components:

Components

1. Client — Mobile/Desktop application that connects to the backend server.
2. Api-Gateway (Auth) — Verifies the user authority and redirects the user to specific service.
3. Load Balancer — Distributes the traffic evenly for fast user experience
4. Read Server — handles read requests like user profile and posts
5. Write Server — handles write requests like upload photo and video and add comments
6. CDN (Content Delivery Network) — CloudFront — Fast Delivery of content to users like photo and video.
7. Cache (REDIS/Memcache) — Stores data temporarily to speed up retrieval and optimize performance.
8. SNS — For every image upload, a notification event is published that is send via SNS.
9. SQS — SQS of different services can subscribe to the upload notification events especially, Feed Generation Service to get the latest feeds.
10. Photo Storage (S3/HDFS) and Photo Storage Replica — Stores photo securely and also create replica of stored photos to enhance redundancy and reliability
11. DataStore (SQL / NoSQL) — Now based on the requirement we will need to use SQL database or NoSQL DB. We can use NoSQL DB like Dynamo DB for storing metadata about uploaded images, user feeds etc. However, relational databases like MySQL can be used for storing the User details.
12. Push Notification to user, if Online — This sends Notification if user is Online.

Services

1. User Service — responsible for user onboarding, login and profile-related actions. The user data is stored in MYSQL and also cached in REDIS. So, now if there is a request for User service, it first checks in REDIS database and if not present then it will check in Mysql database and add it to REDIS before returning to the user. Similar process will follow for new user creation.
   This allows faster performance and maintaining up-to-date information in cache.
2. Post Service — provides API to manage the posts (along with Images and Videos).
3. Comment Service — provides API to add, update, retrieve and delete comments on the posts
4. Like Service — provides API to add likes to the posts or comments, also retrieve the number or likes, and the list of users who have added the likes.
5. Feed Generation Service — This service is responsible for user feed generations and shows the most recent feed for each user. It keeps on listening to the SQS for the post upload events.
6. User Follower Service — This service manages the followers for each user.
7. URL shortener Service — This service will create a short url’s for the image uploads in the S3 buckets and these short url’s will be used to store metadata for the uploaded image in Post Service.

Database Design

Being a Read-Heavy system, with 80% of reads and 20% of writes.
We need to ensure that data store is :

1. Reliable — images and videos uploaded should not be lost.
2. Scalable — Should be able to support millions of uploads.
3. Latency — Should provide Low latency while retrieving images or videos.
4. Replication — Should allow replication and sharding of the database.

Considering the above points, we can use Object storage like S3 for uploading and retrieving images and NoSQL database like DynamoDB (Key-Value) for storing User & Images metadata. Also, we will need cache like REDIS / Memcache to optimize the performance.
Each of these databases should have 3 replicas. Additionally, we need to consider database Sharding strategy.

We will need the following tables:

Overall Functionality

There are 2 major Processes:

1. Synchronous Process — responsible for uploading the images and storing it’s respective metadata and then sending the response back to the user.

When the User logs in the account, he is verified using auth server (api-gateway) and then the post request is redirected to write server, the image is then added to the S3 bucket and the URL Shortener service returns a short url. The photo metadata is stored in photo table and the posts table is updated. Once the post is added, it is send to the feed generation service. This will create personalized feeds for each User. Additionally, an upload post notification is sent through the SNS. The SQS keeps on listening for the new post event notification. The message received by SQS is send to the notification server, which sends the push notifications to the online user’s followers.

2. Asynchronous Process — responsible for persisting user activity in database and triggering the process to pre-compute the feed of followers of non-celebrity users (users with few thousand followers). For Celebrity users, pre-computing of the feed is not done as the process to fan-out the feeds to all the followers is compute and I/O intensive. These feeds for non-celebrity users and celebrity users are combined before showing the followers.

Precompute Feeds
Once a post is created by Non-celebrity user, a message is triggered to the feed generation service which calls the Follower service to get the list of followers. Now this post gets added to the feed of all the followers in the database. These feeds are populated using fan-out service.

Additionally, In order to serve the Feeds to the followers, we can use a combination of Pull and Push mechanism. Wherein for celebrity users, server will be use pull mechanism. While for other users, push mechanism will be used by Server.

Future Enhancements

1. Enhanced security for user photos and videos.
2. Searching based on various attributes like title, tags, captions etc.
3. Separate service for videos to optimize performance.