Understanding Annotation Graphs
Providing a visual way to explore user-authored annotations.
Annotating is the process of associating extra information to a particular point or aspect of data. In visualization, since the data is typically read-only, it is one of the few ways to change a visualization. This process of annotation helps analysts document key observations, derive insights, and easily communicate findings with others. Prior research has identified this process as one of the key steps in enabling iterative visual analysis of data. Currently, a lot of data analysis tools support annotations, but they present these user-authored comments in a linear or textual format (as lists, for example). This approach makes it hard for users to analyze data because data sensemaking is a complex, non-linear process that requires analysts to navigate through details of a dataset in a meandering fashion, often revisiting different data segments to generate insights and discover new trends and patterns.
Annotation graphs were developed to cope with this complex exploration process by visualizing annotations dynamically, using an interactive graph that provides a visual summary of all user-authored annotations from different perspectives.
Creating a Specialized Tool for Annotation
As annotation graphs can’t be evaluated in isolation, the authors first developed a system called C8 that supports creating annotations during exploratory analysis of a dataset. This system was designed following the principles of ESDA, a framework proposed for analyzing observational data recorded in HCI studies. The prototype of the system was then tested with three HCI researchers conducting experiments to collect and analyze large quantities of observational data (such as touch interaction patterns on a trackpad). Based on the testing results and feedback, the final C8 prototype was developed (as shown below).
C8 provides users the ability to select a subset of data in the Data Grid, explore time-series cells in detail in the Timeline View, and further create and browse annotations in the Comments View. In addition, a Context View was created (3D curves, or heatmaps on a trackpad) unique to each of the three researchers’ needs, visualizing their experimental setup. The last part of the tool was the annotation graph view, visualizing annotations and annotation semantics that are continuously generated during data analysis.
Key Features
Annotation graphs are node-link diagrams where the nodes represent annotations in the form of comments, short word tags, or data selections. Links between the nodes describe the content of and relations between data selections, comments, and tags.
These relations (annotation semantics) can be viewed by users from different perspectives:
- Projection-plot positions nodes based on the similarity between nodes, leading to similar comments, tags and selections clustering closer to each other. This helps users get a global picture of the data.
- Slice plot splits nodes based on the type of annotation being represented (data selections, comments, or tags). This would help users understand the relations and connections between different types of nodes.
- Circular-plot allows users to select a particular node (placing it on the center) and observe similar nodes that cluster closer to the center. This would help users understand relations between different nodes and a particular node of interest
Nodes on the annotation graph can also be repositioned manually, pinned, and can be merged together to create groupings, hence providing even more flexibility to the users.
Evaluating Annotation Graphs
The final C8 prototype was evaluated by the three HCI researchers who made use of the system for two weeks to explore experimental data, make annotations, and document interesting findings, insights, and patterns using annotation graphs.
The researchers agreed that the annotation graph was essential in making sense of the data and promoting higher-level insights. The experts made use of the tool in various different ways. Two of the three used the graph as “bookmarks” to revisit where they left off and also used the graph as a way to filter related annotations and data points, hence facilitating easier data exploration. In addition, all the experts liked the flexibility to view the graph from different perspectives along with its ability to manually merge and move nodes.
The feedback also addressed further improvements to annotation graphs such as implementing multiple node selection and hierarchical node merging.
Conclusion
The design and evaluation of C8 clearly show the importance of flexibility when creating annotation graphs. Representing the node-link graph from different perspectives, allowing manual manipulation of the graph, and allowing merging and pinning of nodes allowed users to create custom node-link visualizations that suited their needs.
Based on the feedback, the authors also realized that annotating is itself a very time-consuming process. Future work talks about integrating annotation recommendations within the data exploration tool.
This blog post is based on the following paper:
- Jian Zhao, Michael Glueck, Simon Breslav, Fanny Chevalier, Azam Khan. Annotation Graphs: A Graph-Based Visualization for Meta-Analysis of Data based on User-Authored Annotations. IEEE Transactions on Visualization and Computer Graphics, 2017. [PDF]
