Architecture design of a Data Visualization dashboard
The application that I will be discussing today will not be a walk-through, but more from a system design perspective.
Problem Statement : We are provided with an excel workbook.
All these sheets based on the data that is filled will contribute to a summary sheet. We need to create Data visualizations based on the data filled, to provide some general statistics, which will be section based and based on aggregation of all the forms in each section.
Constraints:
1- Some fields in sheet 1 are related to some fields in sheet 2, sheet 4,etc. and the rest are independent. Hence Data dependency across sheets/ forms needs to be taken care of.
2- Dividing the app into independent sections so that we add or remove components without affecting other parts of the application.
3- Providing CRUD( Create , Read, Update, Delete) functionality in all sheets with data-dependency kept in mind.
4- Limited time- The app needs to be delivered in less than three weeks of time.
What do we want to achieve?
A lot to do and a lot less time 😩
I could have taken up this project on PHP completely since I had some previous experience, but then I decided to separate the API completely(backend) with the View Layer and hence decided to go with Angular 7, as I really wanted to take up a challenge, though it was a lot of risk given the time frame.
What did I use for my Stack?
Angular 7: https://angular.io/as a View layer to provide the front end
PHP for creating the API so that all the data manipulation can be handled independently during testing.
ChartJS : https://www.chartjs.org/ for creating visualization charts. There are other libraries such as d3.js, etc which we can use.
MySQL for database, though we can convert it to a NoSQL structure. Here I used an SQL database because I was aware of the finite number of fields and their relations.
Part 1- Setting up the API
We first work independently on our API without binding it with our View Layer i.e. Angular 7. I personally used LAMP Stack to setup my API.
1- We first create the configurations to connect to our database that is running on localhost.
2- Also we need to add a CORS( https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS )header to our API configuration so that resource sharing is enabled.
3-Once the configurations are ready, we will make separate php files for each function in CRUD (create, read, update , delete). We first create one table per section with the required fields and test out the CRUD calls with Insomnia or Postman(API Testing tools) if the operations are working.
4- We can then move on to creating another table that has dependent values from our previous table to test out the data consistency. We use Foreign keys to fulfill our data consistency constraint. We have a parent table and multiple child tables when we use the Foreign key constraint.
Note: When we delete a row from child table the operation wouldn’t affect the parent table. The only work around that I could think of was, to delete the row from the parent instead of the child table and this way the child tables would also get affected.
5- Using SQL transactions helps us in managing data across mutiple tables, and hence by that logic we create a new record. Our new row would reflect in all other dependent tables.
6- Once our API is ready and tested, we are ready to setup Angular 7 to interact with our API.
Part 2- Making Angular 7 ready
We need to have Npm and Node ready on our machines to setup Angular. Do checkout the versions of Npm and Node required to install Angular 7 as versions might differ based on the version of Angular. Follow this for setup https://angular.io/guide/quickstart
Will get something like this. Our setup is ready.
Part 3- Making Components and Services
In Angular we divide our application into multiple components. These multiple components make use of multiple services according to the need of our application. To explain my point I have attached a flow chart below
We have one root component and multiple section wise components which will be displaying the summary of each section.
The Sectionwise Crud Components will be responsible for the views during the CRUD operations.
Our App component will show us the total summary from different components since it is the root component.
Connecting APIs to AngularThe flow diagram to connect Angular to our API is given. We bind these functions to our Forms to generate Events. Our Event Handlers will manage the HTTP Requests made to our API.
In Angular all the logic is written in typescript. The logic in the typescript is then passed to our html templates. These templates use Directives to render dynamic values/variables that have been passed to our templates. We use these variables inside { } to indicate that these are dynamic values and we should check the typescript file for the values.
Part 4- Using ChartJS to show visuals
In order to get the values for our charts we can make calls to our Crud Service to give us key value pairs according to each section. We can call our Crud Service from any component because it is a part of the root component and hence accessible globally across components. Have a look at the documentation to understand how to create chart objects for different charts and then using those objects in our template: https://www.chartjs.org/docs/latest/
Hence here we are to the end of our architecture design. I have tried my best to not make it confusing for new comers and hence not gone in depth. The following is the Architecture diagram of the whole app. Not so beautiful though 😅 😅
Also, I do believe that this is not the best Architecture design and a lot of redundancy and latency can be taken care of. So let me know if there is something wrongly explained or if you have any suggestions.
Have you read my other articles?
