System Design for Autonomous Vehicle Systems — Part 3
In the third installment of this series, we continue into the communication dynamics between the components outlined in Part 2. We will explore the ideal protocols that ensure seamless interaction between the Telemetry viewer and the BI tools, focusing on finding the most efficient and effective means for their intercommunication. Then we will conclude by summarizing the first version of our overall system design.
Telemetry Viewer
This component allows the end user to query the telemetry data and the data is displayed in real-time in the browser to the user. So if the user wants to see forecasts or trends or make some insights, this component doesn’t do that. This component doesn’t analyze the data, only the BI (Business Intelligence) application will do that.
Application Type
The viewer component will be implemented as a web application, comprising a web server, a web API, and a web browser interface. It will operate on a request-response basis, delivering the data requested by the user. This component is specifically designed as a web app, and will not function as a service, console application, desktop app, or mobile app.
Technology stack
For the web application, we must choose appropriate technologies for both the front-end and back-end.
Continuing with NodeJS for the backend is a logical choice. It offers fast performance and aligns with the team’s existing expertise in JavaScript, making it an efficient and familiar option.
For the frontend, ReactJS stands out as the preferred choice. Given that it is also based on JavaScript, the current team can leverage their JavaScript skills to effectively develop the front-end application. ReactJS’s widespread use and robust capabilities make it a solid choice for building interactive and dynamic user interfaces.
Architecture
Service Interface responsible for exposing the Web APIs, the Business Logic that holds the logic for each API endpoint controller. The Data Access section accesses the MongoDB Data Store to retrieve data. As we have an API for this component, part of our task involves designing the API.
The detailed design of the API will not be covered in this article; it will be addressed in a separate piece. To stay updated on when this new article or video is released, make sure to follow my channels.
Redundancy
To handle the heavy load on this service and in case there is a crash we simply put a Load Balancer in front of the Telemetry Viewer service.
Same classic approach, we handle the load by distributing it equally among all instances. So if one instance goes down, the other can keep functioning.
BI(Business Intelligence) Application and data warehouse
The final component, the Business Intelligence (BI) application, is responsible for analyzing telemetry data and generating customized reports, including data trends and forecasts. This application enables us to answer queries such as the number of cars that broke down in the previous month, among other insights.
BI application is always based on an existing tool, we are never going to develop our own BI tool. There are great BI applications, such as Microsoft PowerBI, Sisense, etc. We can choose the best BI tool by looking at market reports. As of January 2023, Microsoft PowerBI is the leader in BI tools.
As solution architects, designing BI tools and selecting the Data Warehouse for BI tools is not part of our work scope. SWV can hire a BI expert for this process and we will work together with them on the specifics.
So since this is just an example company, we will go with Power BI tool and Azure SQL Data warehouse for the data warehouse.
System Diagram
As we reach the end of this system design process, let’s take a moment to summarize the key components and architecture we have outlined so far.
Logic Diagram
Here’s an overview of the logic diagram for our system: The Telemetry Gateway is responsible for receiving telemetry data from vehicles. This data is then queued by the Telemetry Pipeline for subsequent processing. The Telemetry Processor takes over to validate and process this data, after which it is stored in the Operational Database. Data older than one week is transferred to the Archive Database. For real-time data display, we have the Telemetry Viewer, which pulls information from the operational database. Finally, the BI application, in conjunction with the Data Warehouse, is utilized for data analysis.
Technical Diagram
Here’s a brief overview of the Technical Diagram, outlining the specific technology stack chosen for each component: The Telemetry Gateway, Processor, and Telemetry Viewer are built on NodeJS. For the Telemetry Pipeline, we use Kafka. MongoDB is selected as our Operational Database. The Archive Database is implemented using Azure Cloud Storage, identified as the most cost-effective option. For Business Intelligence(BI), we’ve opted for PowerBI, and Azure SQL Data Warehouse is chosen as our Data Warehousing tool.
Physical Diagram
The summary of the physical diagram is as follows: We have implemented a load balancer for three key components — the Telemetry Gateway, Processor, and Viewer. This load balancer is utilized for distributing the workload evenly and providing redundancy for these components. For the Telemetry Pipeline, we are leveraging Kafka’s built-in high-availability features.
That concludes our case study on SmartWheels Ventures. I hope you found it informative and engaging. While we couldn’t cover every detail due to the scope of this article, like security, data integrity, etc, it still encompassed a broad range of system design topics.
If you’ve enjoyed this case study and are interested in learning more about system design, or if you have a project that requires system design, or any other queries, please don’t hesitate to contact me. You can also follow me on my social media channels, and find contact information on my website: https://anayooleru.com.
I encourage you to share your thoughts in the comments section below. If you found this article helpful, please consider giving it some claps and sharing it with others.
