Instagram System Design Unraveled

— Shubhang Gupta

In the realm of system design, where intricate acronyms like API, MVC, and CDN may initially seem like enigmatic jargon, we embark on a journey to unveil the complex architecture that powers Instagram. Brace yourself, for this is no casual stroll in the park. Designing a system akin to Instagram demands meticulous attention, akin to orchestrating a symphony 🎶 with a multitude of instruments 🎻, each with its unique role. Our approach here shall mirror that of a seasoned engineer, adeptly navigating the labyrinthine world of bytes and databases.

Imagine yourself as a novice explorer, armed with a willingness to learn 📚, yet bearing a perplexed expression rivaling that of a lost kitten 🐾. You delve into a few software architecture books, only to encounter terminology resembling a secret code 🤫 devised by a congregation of tech wizards 🧙‍♂️. You may start to wonder if you’ve inadvertently ventured into a parallel universe, where words such as “scalability” and “load balancing” are woven into the everyday lexicon. This blog, tailored for system design beginners, is designed to transform you into a Jedi Master 🌌 of system architecture, one step at a time. We shall initiate this journey by dissecting the system architecture that underpins Instagram.

Instagram, as you may already know, stands as a social media titan, offering users the platform to share their cherished moments in the form of photos and videos. It’s a realm where likes ❤️, hashtags #️⃣, and filters 📸 reign supreme. However, our objective is not to break the internet but to cultivate an understanding of system design. In this endeavor, we shall craft a simplified version of Instagram, envisioning it as the younger, mischievous sibling, eager to follow in its elder’s footsteps but still learning to tie its shoelaces.

A Brief History

Birth and Humble Beginnings (2010):

• Founded by Kevin Systrom and Mike Krieger in San Francisco.
• Originally called Burbn, focusing on location check-ins and photo sharing.
• Pivoted to focus solely on mobile photo sharing as “Instagram” (a mashup of “instant camera” and “telegram”).
• Launched on iOS in October 2010, gaining 25,000 users on day one.
• Rapidly gained popularity due to its simple interface, photo filters, and focus on mobile experience.

Rise to Popularity (2011–2012):

• Reached 1 million users in less than two months, 10 million in a year.
• Introduced Android app in April 2012, expanding reach significantly.
• Became a cultural phenomenon, popularizing hashtags, influencer marketing, and visual storytelling.
• Acquired by Facebook for $1 billion in April 2012, shortly before Facebook’s IPO.

Global Expansion and Scaling (2013-present):

• Launched web interface in November 2013, followed by dedicated apps for Windows and Fire OS.
• Introduced new features like Direct Messages (2013), Stories & Live Video (2016), and Reels (2020).
• Grew user base to over 2 billion monthly active users by 2022, becoming a top social media platform globally.
• Faced challenges with data privacy, content moderation, and competition from TikTok.
• Parent company Facebook rebranded as Meta in 2021, emphasizing metaverse development.

System Architecture

1. Client-Side Application: Instagram predominantly operates on mobile devices 📱, equipped with native apps for iOS and Android, alongside a web version tailored for browsers. Users engage with Instagram through these client applications.
2. Load Balancers: Incoming user requests are directed to load balancers, entrusted with the responsibility of evenly distributing traffic across multiple servers. This load balancing strategy is pivotal in ensuring high availability and scalability.
3. Web Servers: Instagram’s web servers shoulder the burden of user authentication, profile rendering, and the delivery of web content. These servers proficiently employ web frameworks and technologies to process user requests and generate dynamic web pages.
4. API Servers: The API servers extend endpoints for client applications to interact with the platform. They manage actions such as posting photos, following users, liking posts, and retrieving user data, while also communicating with various backend services to fulfill these requests.
5. Content Delivery: Instagram harnesses the power of Content Delivery Networks (CDNs) to expeditiously serve media content, encompassing images and videos, to users. CDNs cache content at strategically positioned edge locations across the globe, substantially reducing latency and enhancing content delivery speed.
6. Databases: Instagram ingeniously employs a blend of relational and NoSQL databases to oversee its data. Relational databases such as MySQL or PostgreSQL handle structured data, including user profiles and comments. On the other hand, NoSQL databases like Cassandra or MongoDB are reserved for unstructured data, such as user-generated posts and media.
7. Caching Layer: To optimize system performance and alleviate the burden on databases, Instagram embraces caching mechanisms. Frequently accessed data, such as user profiles and recent posts, are diligently cached with the aid of technologies like Memcached or Redis.
8. Message Queues: Message queuing systems like Apache Kafka or RabbitMQ are the architects of asynchronous task management. These queues deftly handle background tasks, ranging from notifications and email delivery to processing user interactions.
9. Search and Discovery: Instagram boasts a robust search and discovery system, empowered by search engines like Elasticsearch. Users can embark on quests to discover accounts, hashtags, and locations, with the system furnishing relevant results.
10. Machine Learning: Machine learning algorithms assume a pivotal role in Instagram’s recommendation systems. These algorithms analyze user behavior and content, offering suggestions for posts, accounts to follow, and advertisements.
11. Content Moderation: Instagram harnesses the power of machine learning and image recognition algorithms for content moderation. These algorithms stand sentinel, diligently detecting and expunging inappropriate or harmful content from the platform.
12. Analytics and Reporting: Instagram bestows users and businesses with analytics and reporting features. Complex algorithms meticulously process user data, resulting in insights on post reach, engagement rates, and follower demographics.
13. Real-Time Features: Instagram caters to real-time features like direct messaging and live video streaming, necessitating real-time communication protocols and a robust infrastructure.
14. Security Measures: An array of security measures, including encryption 🔒, authentication, and access controls, are firmly in place to protect user data and ensure platform security.

📜 Business Requirements

1. Image Upload: Users should have the capability to upload images from their mobile devices 📸, with the system accommodating various image formats and sizes.
2. Like & Comments: Users should be able to express appreciation through likes ❤️ and engage in discussions through comments 💬 on images. They must also have access to view likes and comments on images, and receive notifications when their images are appreciated or commented on.
3. Follow: Users should have the ability to follow other users 👥, and in turn, receive notifications when they are followed by fellow users.
4. News Feed: The system should generate a news feed of images 📰, expertly sorted based on relevancy and freshness.

Technical Requirements

1. The system should gracefully support image formats such as JPEG, PNG, and GIF.
2. The maximum permissible image size should not exceed 20MB.
3. The system must demonstrate robustness by accommodating at least 100,000 concurrent uploads.
4. Users should have the capacity to upload up to 10 images per day 🗓️, with a monthly limit capped at 500 images.
5. The system should efficiently manage at least 1 million concurrent likes and comments.
6. To cater to Instagram’s vast user base, the system should demonstrate its mettle by accommodating at least 1 billion registered users.
7. Users should be able to follow a minimum of 1,000 other users, fostering a rich and engaging social experience.

Instagram imposes certain limits on user activity to safeguard the system from abuse and ensure a pleasant experience for all. While these limits may not affect regular users, business accounts, and social media influencers may need to be mindful of their activity. The limits for a single Instagram user are as follows:

• Posts: There is no official daily limit for posting photos or videos, including Reels. Based on user experiences and reports, regular users can typically post around 20–30 photos or videos per day without encountering issues.
• Uploads: 10 images or videos per day
• Likes: 1,000 likes per day
• Comments: 1,000 comments per day
• Follows: 1,000 follows per hour

UseCases
1. Upload a New Image:

• When a user uploads a new image, the client sends a request to the Instagram server along with the image data.
• The server receives the image data, processes it, and stores it in the appropriate storage system, such as Amazon S3 or a distributed file system.
• Metadata about the image, such as caption, location, and user tags, is stored in a database like MySQL or a NoSQL database.
• Once stored, the image is made accessible to the user’s followers and appears on the user’s profile.

2. Follow Another User:

• To follow another user, the client sends a request to the server with the user ID or username of the target user.
• The server updates the follower and following lists for both users in the database.
• This relationship data allows Instagram to generate personalized feeds and notifications for both the follower and the user being followed.
• When the user posts new content, it will appear in the feed of their followers, including the user who initiated the follow action.

3. Retrieve User News Feed:

• When a user requests their news feed, the client sends a request to the Instagram server.
• The server retrieves the list of users the requesting user follows from the database.
• It then fetches recent posts and stories from those users and sorts them based on timestamps.
• The sorted content is sent back to the client, where it is displayed in the user’s news feed.
• Additionally, the system might use algorithms to personalize the feed based on user interactions, such as likes and comments.

Exceeding any of these limits may result in an error message or a temporary account blockage.

Instagram’s Daily Capacity

• Maximum uploads per day: 10
• Maximum likes per day: 1 million
• Maximum likes per hour: 100,000
• Maximum tags per day: 1,000
• Maximum tags per hour: 100
• Maximum comments per day: 1 million
• Maximum comments per hour: 100,000
• Maximum stories per day: Unlimited
• Maximum concurrent users: 1 billion +
• Maximum concurrent uploads: 100,000
• Maximum concurrent likes and comments: 1 million

Current Status (2024):

• Instagram remains a leading social media platform, but growth has slowed.
• Focuses on features like Reels and shopping to compete with TikTok and e-commerce platforms.
• Addresses concerns about data privacy and mental health with new initiatives and tools.
• Plays a critical role in Meta’s overall strategy, although facing increasing regulatory scrutiny and competition.

Resources:
Instagram Engineering Blog: https://instagram-engineering.com/ (articles on various aspects of Instagram’s infrastructure)
Facebook Engineering Blog: https://engineering.fb.com/ (articles on related technologies used by Facebook and potentially Instagram)

Real-Time Features:
WebSockets: https://developer.mozilla.org/en-US/docs/Web/API/WebSocket (MDN Web Docs)
Server-Sent Events: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events (MDN Web Docs)

From its inception in a small San Francisco office to its global influence as a cultural phenomenon, Instagram has redefined the way we connect, share, and experience the world. Through a symphony of technology and human interaction, it continues to evolve, adapting to new challenges and embracing the future of social networking.