NYCMesh Network Visualization
By: Cosmo Wang, Mark Theeranantachai, Zizhen Song
In collaboration with NYCMesh, we chose to pursue a project focused on building more resilient networks for communities. More specifically, our project aims to provide a visualization tool for NYCMesh to identify alternative paths when a set of nodes is down.
Background and Motivation
The main stakeholders of our project are the volunteers that are maintaining and building out the NYCMesh network. We hope to provide insights on the resiliency of the network and help highlight potential weaknesses that might exist. In turn, we also hope that our project will ultimately result in better service for the community. We understand that access to the internet is important to the success of individuals, and with better insights, we hope to minimize NYCMesh network’s down time.
Design
Our project contains three primary components:
1. Periodic Python Service
The first part is a python service that runs periodically to gather site information of the network, such as nodes information and edges between every two nodes in the network. The service first queries the database for all nodes information, and ping the nodes one by one, gathering all connections between nodes, then either send the edge information to the backend server to cache or store them in the database. Based on our experience, this process takes ~30 minutes to complete.
2. Backend Server
The second part is a backend server that is always running, ready to fulfill frontend requests. The server serves two types requests:
- Requests that can be taken care of inside the server itself. There are currently two requests that fall into this category: one is the request to fetch node information, the other is the request to find an optimal path between the two nodes, potentially with some node marked as disabled.
- Requests that require interaction with the python service. The only request that falls into this category now is the request to ping a node and get all its edges in real time.
3. Frontend Web Application
The last part is a web application that visualizes the information. Below is a screenshot of the application:
As shown in the screenshot, the main feature of the web app is to find a path between two selected nodes. By default, the app shows the primary path between two nodes according to OSPF. Additionally, after displaying the primary path, the user has the option to toggle one or more nodes along the path, to simulate node failure, and asks the server to find a secondary path. The node and edge information used to calculate the primary/secondary path is the cached data in the backend server.
Implementation
Challenges
To ease the development of the project, we assigned each of the team members one part from above to focus on and managed to create a greatly modularized workflow during our development process. However, each of the team members indeed encountered some challenges:
- Python Service & Backend: The most challenging part for the python service and the backend server was about the design. We had a hard time deciding which work should be done in which part and resulted in some wasted work. But we believe that we eventually found the optimal solution, as described above, with the help of our community partner
- Frontend Website: The issue with the frontend website is more about the performance of the application. Rendering ~1000 nodes and many edges in a browser is computation heavy, which can slow down computers with less computation power. Therefore, a lot of work was devoted to optimizing which part to render at which state in order to improve the performance of the application.
Another notable challenge that we encountered was about the site information itself. This problem emerged when we started to put the three pieces of the application together. During the initial development state, a different identifier for nodes in the network was used in each of the three parts of the application:
- Node ID, which is assigned to a user whenever the user signup for the community network, is used in the frontend
- Network Number (NN), which is assigned to a user prior to configuring a router for the user, is used in the backend
- IP Address, which is automatically generated based on the NN when configuring a router for a user, is used in the python service
After multiple meetings with our stakeholders and among the team members, we decided to use NN as the source of truth for node identification throughout the application. Additionally, the python service performs the look up between NN and IP Address, and the frontend performs the look up between NN and Node ID.
Feedbacks
In order to further improve our implementation and orient towards our stakeholders, we reached out to Rob Johnson, who is a member of NYC Mesh, for feedback on our application, focusing on the user interface part. After reviewing our implementation, the stakeholders of the project indicated interest in a more expressive user interface as well as some additional features.
The stakeholders described some use cases of the tool that indicated the need for displaying additional information than what we had. Additional information that our stakeholders were interested in including individual cost of edges, Network Number and IP Address of a node, and a real time view of edges from specific nodes.
Below is a summary of the new features we added and a screenshot of our application after fulfilling the requirements:
- We included individual edge costs, as well as the total cost of the path, inside the top left floating window
- We collected all information we have about a node and displays that information in a single popup window of a node
- We improved our pathfinding algorithm to find the path with the lowest hops when the total edge costs are equal. We also took hidden nodes and bridges into consideration, so the alternative paths will be more accurate.
- Lastly, we also added the ability to request a real time view of edges from a specific node by either requesting cache information on the server or triggering a node-ping script on demand. Since the request involves pinging only one node, the delay is much shorter than the request needed to create the whole graph, allowing for a timely response.
Reflection and Next Steps
Overall, we had a great experience working on this project. Our nicely modularized work had a few hiccups, but were overall very pleasant because of frequent meetings and discussions. Based on our conversations with our stakeholders, our application satisfies their needs and will be useful in practice in the future.
Therefore, the next step for our project is to discuss with our stakeholders about its deployment. We’re currently unsure about how this will happen as it will depend on the type technologies available to use in NYC Mesh.
Credits
Community Partner from NYC Mesh:
Rob Johnson (Special Thanks)
RouterOS API used in our Python service:
https://github.com/DEssMALA/RouterOS_API/tree/ebf1cb34bd4df8701456a97627c56687450e9b24
