System Design — QR code generation
Today, I came across an intriguing task related to designing an application for generating QR codes. I will strive to delve into the project details as comprehensively as possible and explain the nuances that should be considered from both the perspective of backend development, written in Node.js, and frontend development, built using AngularJS.
The task can be summarized as follows:
The user’s requirement is to create QR codes for application assets, identified by a unique ID, and enable scanning these QR codes to open the corresponding application asset in a web browser. The QR codes have not been generated yet, and the application is built using Angular for the frontend and relies on a Node.js server for the backend. The frontend can display up to 10 previously generated QR code images simultaneously.
Your task is to outline how you would implement the generation and scanning of QR codes while considering potential user errors. There are specific constraints: the QR code generation must occur on the backend, and it’s crucial to verify the existence of an asset with the given ID before generating the QR code.
Please provide a detailed solution separately for both the Front End (Angular) and Back End (Node.js) aspects. Essentially, you need to address the QR code challenge from both the user interface and server perspectives, with a focus on text-based solutions, without the need for actual code implementation.
## QR Codes Introduction
Before diving into system design let’s answer on the question what QR codes are and what kind of data it can store.
A QR code, short for Quick Response code, is a 2D barcode that has transformed the way we access information in the digital era. QR codes can store diverse data, including website links, contact information, and product details. They offer a seamless and efficient means of sharing and accessing information through a simple smartphone scan. In this article, we’ll explore the versatility and applications of QR codes, showcasing how they streamline various aspects of our lives.
In general, you can fit a substantial amount of data into a QR code (long URLs with all required query parameters), but it’s best not to overdo it. From my perspective, it’s wiser to rely more on a database, which is designed for handling such extensive data. Besides that, compare these two generated QR codes. Let’s be honest, the second one is much more beautiful.
## Functional Requirements
Despite our current task at hand, it’s essential to clarify some details for a comprehensive understanding.
1. Do we need a blob storage for files? If so, we must also consider metadata storage for these files.
2. Given the ability to upload up to 10 images at once on the client, pagination needs to be factored in.
3. When discussing QR code generation and scanning, it’s akin to a simple URL shortener task.
4. Since we lack user quantity information, let’s aim for a solution resembling the extensible models seen in popular SaaS platforms.
## Backend
### Rough Estimates
Here, I’ll provide rough estimates based on calculations and figures from the link shortener generator:
- **100,000,000 records per day**
- **1160 Requests Per Second (RPS)** considering 100,000,000 records spread over 24 hours and 3600 seconds.
- Assuming a read-to-write ratio of 10:1, we have **11,600 RPS** for reads.
- If the service operates for 10 years, we need to support storage for **365 billion records**.
- Assuming the average length of a saved URL is 100 characters.
- Storage requirements for the next 10 years: **365 TB** for links alone, not including blob storage.
Additionally, for the server:
- Each connection requires approximately **70 KB**.
- The server can handle around **40,000 concurrent connections** using Fastify (in ideal world, but still performance testing should be done)
Considering this information, we can make educated assumptions about when we will need to scale our infrastructure.
To summarize, these estimates help us anticipate the significant storage and traffic demands our system will face over the next decade, allowing us to plan for scalability accordingly.
### Backend Considerations
So, apart from hardware limitations, what should we consider on the backend?
#### Asset Existence Check
Checking the existence of an asset. While it may enhance system stability, it’s not strictly necessary. However, considering security, we’ll need to ensure resource ownership (not just the link but the file ID we attach).
#### Uniqueness of the QR code
As I was thinking about this, it occurred to me that we’ll likely need a mechanism to generate unique QR codes for the same data. In this case, the backend should generate a unique identifier. Using UUIDs is a reasonable approach for generating unique identifiers in distributed systems.
Ensuring the uniqueness of QR codes generated by our link shortening service is pivotal to avoid collisions and maintain a seamless user experience. One implementation approach involves hashing the long URL. When a user submits a long URL, the service can create a unique hash of the URL using a cryptographic hashing algorithm like SHA-256. This hash serves as the core of the QR code, ensuring that each long URL maps to a distinct QR code.
### Endpoints implementations
In terms of implemented endpoints, we’ll need just a couple:
1. An endpoint for QR code generation.
2. And endpoint for QR code listing for owner
1. Keyset or offset pagination, security considerations and everything here are almost obvious, so I will not dive deep in it.
3. An endpoint that takes a short URL and redirects users to a longer URL using a 301 or 302 status code.
In the context of implementing a link shortening service, the choice between using HTTP status codes 301 (Moved Permanently) and 302 (Found) is crucial and depends on the service’s specific goals. If the aim is to incorporate analytics into the service, the preferred option is to utilize the 301 status code. A 301 redirect indicates a permanent move of the resource to a new location, and search engines, browsers, and web crawlers interpret it as a signal to update their indexes accordingly. This permanence allows link shortening service providers to track user interactions effectively, gathering data on click-through rates, user demographics, and geographic location. By maintaining the permanence of the redirect, analytics data can be collected consistently over time, providing valuable insights into user behavior.
On the other hand, if the primary focus is on server optimization, using the 302 status code is a suitable choice. A 302 redirect indicates a temporary move or a redirection that might change in the future. It does not prompt search engines to update their indexes as aggressively as a 301 redirect does. This can be beneficial in cases where server resources need to be conserved, as 302 redirects require less processing overhead and may result in lower server load. However, it’s essential to note that the temporary nature of 302 redirects may not be ideal for tracking analytics data reliably, as they do not guarantee long-term consistency in user interactions.
In summary, the choice between HTTP status codes 301 and 302 in a link shortening service hinges on the specific objectives of the service. If comprehensive analytics and long-term tracking are a priority, 301 redirects are preferable. Conversely, if server optimization and resource conservation take precedence, 302 redirects offer a suitable solution. It’s essential to align the choice of status code with the overarching goals of the service to ensure optimal performance and functionality.
#### Other
While we’ve discussed dynamic QR code generation and link shortening, in theory, the task might be achievable with static generation based on the initial problem statement.
It’s important to bear in mind that our system should be designed with scalability in mind. As our user base expands, horizontal scaling may become a key factor in ensuring our system’s performance and responsiveness
## Frontend
### QR Code Generation
From the client’s perspective, implementing the required functionality is relatively straightforward. However, there are still important considerations to keep in mind.
Starting with QR code generation, we realize that we need to implement an interface that allows users to upload applications into the system. While this wasn’t covered in our system design, at a high level, it involves storing metadata in a database and the file itself in a blob storage.
With knowledge of the file’s metadata, users can select them from a list and send a request to a prepared server for QR code generation.
### QR Code Listing
As previously mentioned in the requirements, I noted the limitation of 10 QR codes per page, which led to an assumption. Therefore, the need to implement this functionality on the client side arises. Pagination, of course, will be handled on the server side.
Regarding the client-side implementation, it would be prudent to consider a possible implementation of a virtual list for future scalability. However, for now, a conventional table or list can suffice without negatively impacting performance.
### QR Code Reading
We’ve delved into this topic while working on server endpoints, and in general, we have a solid understanding of how it operates. Let’s recap briefly:
1. The user scans a QR code using their phone.
2. Instantly, they are redirected to a browser on our server’s domain, where a website is hosted.
3. This website queries the database, retrieves the long URL generated earlier, and responds to the client using HTTP status codes 301 or 302.
4. Consequently, the user is redirected to a page responsible for displaying the attached asset we selected earlier.
I see several potential implementations here to consider. Either we redirect the user to a URL that includes the document type in the path (for example, it could be like [https://example.com/asset/unique-id](https://example.com/asset/unique-id)). However, this approach may limit our ability to dynamically update content. Alternatively, we could redirect the user directly via a unique identifier, and then, on the server, determine the content type. Depending on this, we can serve different displays to the client. For assets, it could be images stored in blob storage, and for lists, it could be the actual links themselves.