Evaluating Interactive Graphical Encodings for Data Visualization

What is a graphical encoding?

Data representation is one of the main components of data visualization. The fundamental focus of data representation is mapping from data values to graphical representations. Visualization designers use elementary graphical units called “graphical encodings” to map data to graphical representation. Consider a case in which we visualize two numerical values using two bars with different lengths. Here, length is the primary encoding variable used to map the data values. Alternatively, angle is the primary graphical encoding for a pie chart.

Why understanding graphical encodings matters in data visualization?

A great deal of prior research has investigated how graphical encodings such as position, length, area, shape, and color impact the effectiveness of data visualizations. These studies often measured ability of viewers to accurately perceive the data values encoded using graphical encodings.

Fig 1. Elementary graphical encodings studied by Cleveland and McGill (these images were recreated and adapted based on the original paper by Cleveland and McGill).

One of the most well-known studies investigating the effectiveness of graphical encodings is that by Cleveland and McGill. The study tested the user perception of 10 elementary graphical encodings. Participants were asked to visually compare values of two marks (e.g., two bars of different lengths) and estimate what percentage the smaller value was of the larger. They used the results to rank the graphical encodings; one elementary graphical encoding is taken to be more accurate than another if it leads to human judgments that are closer to the actual encoded values.

Understanding how people perceive different graphical encodings enables designers to optimize their visualizations and aids the design of automatic presentation software.

What are interactive graphical encodings?

More recently there has been an increasing trend in information visualization and visual analytics to allow users to directly interact with graphical encodings. We define interactive graphical encodings as an elementary graphical encoding that can be directly manipulated or adjusted.

Fig 2. Some of the interactions supported by DimpVis. Direct manipulation of the length of a bar, angle of a pie chart, and positions of the data points in a scatterplot. This figure is used by permission.

For example, DimpVis is a recent system that allows users to directly interact with the length, angle and position of the graphical encodings used in visual representations, as a means for temporal navigation (See Fig 2). In DimpVis, users can adjust the length of a bar to see its value at different moments in time. For instance, to check if at any point in time the value associated with a bar is half its current value, the user can drag the bar vertically downwards to compare its values at different points in time.

Fig 3. In the Visualization by Demonstration paradigm, a user directly interacts with a point by making its size larger to demonstrate the interest in generating a visualization in which this point, and points like this, are larger. In response, the system extract data attributes that can be mapped to size and suggest them.

Visualization by Demonstration is another example in which users can directly interact with graphical encodings to provide visual demonstrations of incremental changes to the data representation. For example, the user makes the size of a data point two times larger to demonstrate interest in generating a visualization in which this point and similar points are classified together and shown larger than other data points. In response, the system solves for data attributes that can be mapped to size and suggests the attributes.

Understanding interactive graphical encodings in data visualization

As described earlier, a great deal of previous research has contributed towards an understanding of perception of different static graphical encodings. We believe the field lacks the knowledge of how different graphical encodings can serve as the basis for user interaction. In 2015, we decided to conduct a study where we measure the effectiveness of 12 different interactive graphical encodings for magnitude production tasks (e.g., change the value of the interactive graphical encoding to x% of its current value). Our goal was to understand the effectiveness of user interaction with these graphical encodings rather than the how well users perceive their encoded values.

Fig 4. The 12 interactive graphical encodings assessed in this study, designed based on seven common elementary graphical encodings used in data visualization: distance, position, length, angle, curvature, shading, and area.
Figure 5. A screenshot of one of the trials used in this experiment. This specific trial asks a user to adjust the angle encoding to 200% of the current value.

We conducted an online study where participants performed seven trials for each of the 12 versions of interactive graphical encodings, and each trial had a different target value (25%, 50%, 75%, 125%, 150%, 175%, and 200%). Participants performed 84 tasks (12 interactive graphical encodings × 7 trials) with randomized task order. Each interactive graphical encoding was accompanied by instructions that required the participant to adjust the interactive graphical encoding to a target value. A target value is a certain percentage that we asked each participant to adjust the interactive encoding to. For example, for the angle encoding, we asked participants to adjust the inner angle between the two lines 200% of its current value. Participants could adjust the graphical encodings’ values by directly manipulating them, as described previously.

Accuracy and interaction times differ among interactive graphical encodings

To analyze the data collected from our user study, we first measured both interaction time and accuracy for each trial. Interaction time was measured by computing the total time each participant spent interacting with a graphical encoding.

Our results indicated that position encoding has the best and shading has the worst accuracy. Accuracy of position was significantly better than all other interactive graphical encodings. Length, distance, and angle were interpreted with similar accuracy. We also found that shading was significantly less accurate than all other encodings. Moreover, area and curve fall somewhere in the middle in terms of the accuracy ranking.

Participants had the fastest interaction times using length, position, and angle, respectively. Although results of pairwise comparisons did not show significant difference among the three interactive graphical encodings, they were significantly faster than area, curve, distance, and shading. Curve, distance, and area were in the middle in terms of time.

Fig 6. Ranking of the interactive graphical encodings based on completion accuracy and interaction time. Rows indicate significant differences between encodings. Bolded encodings highlight deviations in rankings from Cleveland and McGill’s experiment.

We also compared our ranking of the interactive graphical encodings from the results of the study by Cleveland and McGill. In general, our ranking follows that of the prior studies, with the exception of our study observing a significant difference between length and angle in terms of accuracy. Fig 6 shows rankings of the interactive graphical encodings based on the different metrics assessed in our paper alongside rankings of graphical encodings provided by Cleveland and McGill. In each column, interactive graphical encodings are ranked from best to worse according to performance in each metric. For example, position has the best and shading has the worst accuracy in our study. An explanation for this similarity may be that manipulation and perception are not mutually exclusive, and input from perception continually influences interaction. Thus, the performance of interaction with an encoding might be connected to the perception of the encoding itself. If an encoding supports sheer perception well, it would also support interactivity well.

Bias Analysis

We conducted chi-squared tests to check whether user interactions with different encodings were biased towards overestimation or underestimation. For each encoding, we ran separate tests for trials asking for increasing values and for those requiring decreasing values. For these tests, we excluded responses with exact accuracy for the level of precision in data collection.

The results show that there are significant effects in responses being biased towards either over or underestimation — particularly for responses where participants were asked to decrease an encoding’s value. When participants were asked to increase the values, significant response biases were observed for 3 out of the 7 encodings (shading, curve, area) — the encodings having lowest overall accuracy. For example, area is the only encoding with an underestimation bias for increasing the value. This could be explained by the fact that increasing the value in area is not a linear increase but a squared increase. Among all encodings, shading has the highest skew towards under or over estimation, which is likely related to the ineffectiveness of the encoding. For responses where participants were asked to decrease the value of the shading encoding, all the responses underestimated the expected value. While these results indicate that bias is important when exploring the effectiveness of interactive graphical encodings, further studies are needed to fully understand what causes these biases.

How to incorporate the interactive graphical encodings?

Based on our findings, we suggest following guidelines.

Making encodings interactive requires careful design considerations: Not every encoding used in a given visualization needs to be interactive. In cases where the chosen visual representation requires the use of an encoding with low performance, perhaps the use of traditional control panels for interaction is the better design decision. For example, visual representations that use shading or area as the primary method to encode data may be augmented with control panels to control the filtering or querying rather than embedded interaction (e.g., geospatial choropleth maps). Instead, visual representations that use effective encodings lend themselves better to incorporating interactivity directly on the encoding.

Add visual cues to help users understand the interactivity latent in the encodings: One way to improve the performance of the interactive encodings is to make the affordances more salient to the users. For example, we could provide an animation upon hovering on each encoding to show how to interact with it.

Provide additional feedback if accuracy is important: Providing additional feedback might be helpful to improve the performance of specific encodings. For example, during embedded interaction with shading, interaction performance might be improved by also showing exact values via textual overlay. Additionally, we could highlight the aspects of the encodings that contribute to the value change. For example, for angular encodings, we could highlight the angle subtended or the height between the two arcs. Similarly, for area encodings, we could highlight the width and height of the square to show the squared value. While we did not test the effectiveness of such potential design improvements in our study, these considerations could be of interest for future design and evaluation efforts.

To learn more about our work, please check out our paper.

This post was written by Bahador Saket. This work is in collaboration with Arjun Srinivasan, Eric D. Ragan, and Alex Endert.

A single golf clap? Or a long standing ovation?

By clapping more or less, you can signal to us which stories really stand out.