On the Inside of Insight

Niklas Elmqvist
May 24, 2019 · 4 min read

Capturing insights from data analysis in its full context and detail.

Organizing your insights from data analysis can often be a tricky endeavor. (Photo by Vojtech Okenka from Pexels.)

Making sense of data to understand a difficult problem is often a chaotic process. Consider reading reviews to decide on which TV to buy, finding the right resort for your family vacation, or selecting which university to send your kids to. We use the term sensemaking to denote this process of understanding data, and insight to refer to the useful pieces of information gleaned in the process. Sensemaking for insight is even more chaotic in professional settings: consider a cancer board deciding on treatment for a patient, a civil engineering team planning a difficult bridge construction, or an investigative journalist reviewing interview transcripts to piece together a breaking story. Lots of data, few guidelines, and vague goals.

What these sensemaking scenarios often amount to is a meandering and repetitive process of organizing data, drawing conclusions, and making educated guesses. Sometimes the analyst will have an idea of what is going on in the data, and will need to dive deep to find evidence that supports this idea (or not). Sometimes the analyst will instead have to start from the data itself, drawing conclusions by combining a few facts at a time to gradually build a bigger picture. To make things even more challenging, real-world situations often involve many people, just like in the cancer board, family trip planning, or bridge construction example above.

Interactive software can make sensemaking significantly more effective, but most current tools are not flexible enough to support the chaotic nature of true sensemaking tasks. For example, the commercial data analysis tool Tableau has powerful visualization and data management capabilities, but lacks mechanisms to capture, organize, and disseminate insights as part of the analysis process. Even the new generation of computational interactive notebooks, such as Jupyter and Observable, do not easily support transitioning between data exploration and analysis as well as communicating insights to others.

In our EuroVis 2019 paper, we present InsideInsights, a new sensemaking tool to better support chaotic, unstructured, and meandering data analysis processes performed in teams of analysts. The new tool runs in your web browser without the need for a separate download. Furthermore, it builds on our existing Vistrates platform for creating shareable visualizations that can be accessed concurrently by multiple people and on many devices.

At its core, InsideInsights is essentially a way for analysts to store insights and arrange them in a hierarchy as they make progress in their thinking. These insights can either be low-level findings drawn from specific data and specific visualizations of that data; for example, a business analyst may find an anomaly in a time-series plot of stock market indices. Or they may be high-level ideas that the analyst has about the data; for example, a police sergeant speculating that vandalism is tied to home football games in a university town. In both cases, these insights can be recorded in a hierarchy, where they then can be grouped together and arranged. For low-level insights, the analyst will typically group related insights into bigger themes; for example, many market anomalies indicating a coming trend. For expressing high-level ideas, the analyst will instead look for evidence in the data to support the idea; the sergeant finding all instances of vandalism and connecting them to the football schedule.

Insight hierarchy in InsideInsights (left), with the insights in the hierarchy becoming slides in a presentation (right).

An insight hierarchy can also be used as a presentation to remember your own work from the previous day, to onboard other members of the analyst team, or to brief a manager or visitor on current findings. When used for presentations in this way, the insights become slides that refer to specific data points and visualizations, and navigating the hierarchy switches the slides. In the picture below, an analyst has used InsideInsights to analyze a business event dataset to understand the behavior of different companies over time. The left side of the image shows the insight hierarchy, where each insight has a textual description and has been organized into the tree structure. On the right side are the visualization slides connected to each individual insight. The user is currently viewing the grey insight labeled “(3)”.

Insight hierarchy and slides for a business event analysis.

The key difference about using InsideInsights for presentation compared to traditional such as PowerPoint and Keynote is that navigating the insight hierarchy essentially means that, besides navigating to the next and previous slides, you can also go “down” and “up” to get more or less detail. Going down in the hierarchy means visiting the children of an insight to see the lower-level insights that support it, whereas going up means seeing the conclusion that can be drawn from it. The picture below illustrates this idea.

Navigating an insight hierarchy: going left and right visits siblings on the same level of the tree, whereas going up visits the parent and children of the insight.

We look forward to seeing additional ways people can use InsideInsights to find and present interesting insights from their data. Meanwhile, here is a video showing the tool in action:

InsideInsights will be presented at EuroVis 2019 in Porto, Portugal on June 3–7, 2019. Here is the full citation:

  • Andreas Mathisen, Tom Horak, Clemens Nylandsted Klokmose, Kaj Grønbæk, Niklas Elmqvist. InsideInsights: Integrating Data-Driven Reporting in Collaborative Visual Analytics. Computer Graphics Forum (Proc. IEEE EuroVis 2019), 2019. (PDF)

Sparks of Innovation: Stories from the HCIL

Research at the Human-Computer Interaction Laboratory at…

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store