What Does the Chart Say?
Harnessing the power of textual references to charts.
There is a prevalence of chart-to-text references in data-driven documents such as news articles, online blogs, and academic papers. In a recent paper presented at IEEE VIS 2021, researchers studied such pairings of chart and text to communicate complex data. When done correctly, using a chart-text reference can summarize your data and provide context to the data which enables narrative visualization and data-driven storytelling, the ultimate goal for any data driven document.
Synthesizing the capabilities of text and charts in the most optimal way can be difficult because users must shift their attention between the text and the chart. Depending on the reader’s cognitive abilities, there can be information that was not understood by the reader. This phenomenon is known as split-attention in cognitive load theory, and it requires readers to use their working memory to understand, which in turn can impact learning. The authors, echoing many researchers before them, believe that the solution lies in better linking the text and the charts it refers to, or by using visual highlighting more effectively. Unfortunately, this often requires advanced programming.
Enter Kori.
Kori is helps the visualization designer to combine text and charts through so-called “interactive references.” In order to build this application, the authors studied text-chart pairings to find similarities between natural language and selection operations on charts, such as selecting points and intervals. From this study they derived an approach for in-text references that correspond to such typical selection operations.
Minimal references are the most basic interactive references: a reference to an individual item in the data, a related group of items, or a range. Multiple minimal references can then be combined to make higher-order or parent references. Taking a look at the image above, focus on phrase A: “One group is comprised of Spain, Italy, the UK, and Ireland”. The minimal references in this situation are the individual countries. Together they make up the building blocks of the parent reference, which is that of the four countries. Now let’s focus on phrase B: “The national median disposable income in these four countries ranged from $30,000 to $39,000 in 2010 and the middle-class shares ranged from 64% to 69%…” The minimal references in this situation are “four countries”, “ranged from $30,000 to $39,000”, “ranged from 64% to 69%”.
After Kori has identified minimal and parent references in the text, it suggests to the designer how to best create a connection between the text and chart. All parent references are either a union of like minimal references or an intersection of minimal references that refer to differing variables. In the examples we addressed above, Kori would understand phrase A as a union and phrase B as an intersection.
Ease of use was another important design principle for the Kori system. The developer’s goal was to create a program that was accessible to those who are not expert programmers. This design principle also manifested in other areas. The developers were intentional in making Kori offer suggestions that the designer can either accept or reject. This is to avoid false positive references to the chart and to avoid too many suggestions. The developers wanted to give designers the ability to also create their own references. Finally the developers wanted Kori to assist and not distract from the ultimate goal of the app, which is to create content while assistance for linking text and charts blends in smoothly within the app.
In my view, designing accessible ways for people to understand data visualization is always a great idea, especially because the amount of data we consume every day is steadily increasing. A tool like this that not only helps improve cognitive understanding of data but also improves cognitive understanding of how to build data-driven documents is quite a brilliant idea.
References
- Shahid Latif, Zheng Zhou, Yoon Kim, Fabian Beck, Nam Wook Kim. Kori: Interactive Synthesis of Text and Charts in Data Documents. IEEE VIS 2021.
