Design Natural and Fluid Interactions for Visualization Tools

Using direct manipulation as a method for interaction in visualization.

From Apache Echarts.

Visualization can help users graphically process information through visual visual representations, which is useful for large datasets. However, novice users may feel a little lost with visualization tools due to the lack of knowledge for visual representations. In this situation, many users would likely try every function provided by the visualization tool. This is obviously error-prone and inefficient. How can visualization tools provide users with better interaction or even improve decision making through interaction?

Direct Manipulation in Visualization Tools

Currently, designers and researchers show great interest in applying direct manipulation methods to the interfaces of visualization tools. Direct manipulation refers to people directly acting on their objects. For example, changing a vector on the display by dragging it to interact with axis, length, and positions of visual presentations. Direct manipulation helps users edit more easily and quickly in visualization. Moreover, people will be more engaged across the visualization process using direct manipulation.

The followings are three main types of direct manipulate operation in visualization.

• Position: Users can directly change the position of the visualization (such as bar chart, histogram, 2D scatterplot, etc) by dragging and dropping.
• Color: Users can select a color in a color picker (close to the visual mark) and change the color of visualization.
• Size: Users can drag the outline/ border of the visualization to change the visualization’s size (height and width).

While there are different types of direct manipulation in the visualization field, designers need to decide which operation should be provided to users in the visualization tools. It is also important to decide under which circumstances those provided operations will be invoked in visualization tools.

Deriving Empirical Guidelines

Deriving optimal interaction methods is challenging without empirical guidelines. Therefore, in a paper published at IEEE InfoVis 2019, Bahador Saket and his collaborators conducted experiments to empirically study designs for better direct manipulation strategies on those tools.

In their first pilot study, researchers asked three participants to verbally explain how they would perform a series of visualization tasks using direct manipulation of graphical encodings on a paper-based visual presentation. The challenge of the study was that participants needed to imagine their interactive operations while they were using an interface that was not interactive. Participants were confused and found it was hard to explain their strategies when using the tool.

In the second pilot study, researchers found that participants were able to explain their thoughts when performing the majority of the operations using direct manipulation of graphical encodings. This pilot study indicated to researchers that using a web-based dataset was appropriate and they continued using it in their formal study.

In the third study, researchers first categorized direct manipulation visualizations, including encoding types, operations, and datasets. Then they explained these categorized operations to participants. After getting familiar with those operations, participants were asked to manipulate the visual presentation using provided operations and tell researchers the reason why they decided to use that specific operation.

Some testing operation examples are shown below:

From the final study, researchers found that most operations were adopted in different degrees by participants. Also, participants used the same operations but with different strategies. Researchers compared the relationship between operations and strategies participants used in the study and identified four high-level approaches that participants used to manipulate encodings to invoke an operation.

The four high-level approaches are exemplification, declaration, instrumentation, and selection. Exemplification and declaration are approaches that would be deployed for only one or two visual marks during the visualization, while users would use more instrumentation and selection approaches if they need to perform repetitive actions on graphical encodings rapidly.

Conclusion

Direct manipulation is a powerful method in interaction design. It can be a powerful addition to visualization tools. Researchers have the following suggestions for how to achieve this:

1. The mechanism people use to manipulate visualizations is not complex, and majority direct strategies often rely on a single type of direct manipulating operations.
2. No single strategy can universally perform an operation in visualization.
3. Some strategies can be consistent in performing different operations.
4. Avoid implementing support for strategies that different people would use for performing different operations.

This post is based on the following paper:

• Bahador Saket, Samuel Huron, Charles Perin, Alex Endert. Investigating Direct Manipulation of Graphical Encodings as a Method for User Interaction. IEEE Transactions on Visualization and Computer Graphics (TVCG / Proc. of Infovis ‘19). IEEE