Case Study 01: Making Visualizations Resonate with How Researchers Think

Planetary Micro X-Ray Fluorescence Spectrometry

Context

The mars 2020 rover PIXL instrument is designed to detect signs of ancient life in the rock formations of the red planet. Organic processes leave markings, patterns, chemical evidence which cannot occur without biological mediation. Geologists on earth use these same techniques in their search for the oldest direct evidence of life on our planet. Similar microXRF instruments are used in both contexts to investigate the spatial relationships between elements and minerals (solid, pure, chemical compounds) which to the expert eye can tell the story of how a rock was formed, and thus determine if biologic mediation must have occured.

Carrying this type of investigation out through the instruments of a planetary rover confers additional challenges. On Earth, additional tests may be carried out to dis-confirm hypotheses as needed. Working on a different planet, the researchers ability to interact with the rock formation is restricted, putting additional emphasis on the importance of the visual analysis of spectroscopic data.

Mission

Scientists at the NASA Jet Propulsion Lab need to be able to visually assess microXRF data sets in order to determine if biologic mediation has occurred in a given soil or rock sample. Currently the team extracts the concentration values for each element present in the sample, and then renders each element map in a distinctive color. The expert then reads the geologic history of the rock across all of these maps.

Limitations and Challenges

Simple visual solutions such as this can be helpful in scientific decision making, but are of limited utility in addressing more complex cases. The prototypical simple case is illustrated below. One known form of geologic evidence of biologic mediation is called Oolite. This is a sedimentary rock formed of small concentrically layered grains called Ooids. This distinct radial crystal can be formed through accretion of microbial biofilms, which can be observed with microXRF instruments.

The concentric spatial distribution of elements within the Oolite make this a simple problem to address with visual analysis. Separate maps representing the concentration of each element are generated and given a distinct color. These can be used separately in a small multiples style comparison, or they can be overlaid by mapping the concentration from color to alpha as opposed to color to black. Because this particular sample has little in the way of color overlap, or color mixing due to complex mineralogy, the resulting image is clear and the story simple to comprehend. Unfortunately most samples are not like this, and more complex visual strategies must be applied to comprehend them.

A different and more challenging sample is depicted below. This is an Earth rock which was once thought to be evidence of ancient life. Scientists on the PIXL team now think differently. Their visual analysis methodology is based entirely on small multiples cross comparison because the amount of co-located elements and minerals makes overlay visualizations ineffective.

Problem

In this sample, all seven of the depicted elements overlap in the upper region, an attempt to understand this region through overlaying of color element maps would be ineffective. The illustration below demonstrates the difficulties inherent in attempting to disambiguate color mixing of several distinct colors. Although each color is legible on its own, the various mixtures are impossible to characterize with the level of confidence needed to support a scientific conclusion. However, working from an abstract analysis of the pure values is equally ineffective because high-level patterns are not highlighted in any way.

Approach

Part of a scientific visualization designer’s role is to provide novel solutions for circumstances such as these. An effective method of moving past this type of challenge, is to understand the researchers deeper need: what are they trying to know or understand? Although scientists have an established way of working, that way may not be the most effective, especially if it is reflective of previous software limitations and the force of habit.

The JPL team’s approach of small multiples and opacity overlays came about because it was the only approach afforded by the analysis software that they use. Continual use of these limited products generated an industry-wide habit of thinking, which the researchers expressed to us as their need. However, there was a deeper layer to discover below this.

In this case, we were able to run co-design sessions in which the researchers revealed that they wanted to understand where minerals were located spatially, to see the mineral’s spatial distribution clearly. As such, color was an un-necessary element of the visualization. The visual solution which we offered allows researchers to input the combination of elements which make up a given mineral, and receive a black and white visualization which gives at each pixel a binary answer to the question, are all requested elements present at this location? What we discovered was that the researchers thought that they needed to see multiple element maps overlaid, but what they actually needed was to see where minerals were present.

Visual Solution

This view is simpler, it culls unnecessary detail from the representation in order to represent the answer to precisely what the researcher wants to understand. This has a downstream affect on the researcher’s workflow: the visualization more clearly tracks or resonates with how the researchers think about the sample and its formation history. In the previous visualization, researchers spent much of their cognitive effort on decoding the color blocks that they were examining. In this visualization, on each card the researcher can see precisely the location and distribution of each element or mineral.

Design Decisions

The interface that this visualization lives in is black with light type and bright visualizations. I took this from radiology interfaces that I have worked on in the past. Radiologists are visual pattern analysts who see subtle and intricate differences in shades of gray. These shades represent the evidence of disease in the human body and a radiologist is significantly responsible for how a disease will be treated, they must see carefully all of the tiny subtle differences in color from an x-ray or MRI, the patient’s health depends on this. In order to support this careful seeing, radiologic interfaces are dark, which removes excess light that would otherwise constrict the pupil. With their pupils open wide to catch all of the detail, the bright visualizations become the focus of their analytic attention. The UI for this visual tool takes a similar approach because the PIXL scientists are doing something very similar, grayscale investigation of subtle patterns which represent a complex story to be discovered.

Another critical decision was to remove the color from each element map. This color originally played the role of helping the scientists to distinguish each element in opacity overlays. Additionally the team developed a common chromatic language around each element: for example all scientists on the team agreed to represent iron with red, this helps in team cohesion and consensus building. Since we removed the opacity overlay technique, each element map could be reduced to grayscale, which is a more perceptually consistent representation. Because of the contingencies of human eyes and visual cortex perception, a dark blue pixel will appear darker than a yellow pixel even if they are at the same light value. By removing color from the map, we eliminated this challenge from the scientists daily workflow. In order to preserve the team chromatic language, each map card has a single color dot next to the element letters, allowing the team to maintain their same cognitive connection of element to color.

One further challenge is the difficulty recognizing features across element maps. To overcome this, we only allow horizontal spatial comparison of maps and intersections. This way all distinct features will be at the same Y height in space, supporting side-to-side scanning tasks. To further clarify features, we created a feature trace method which allows the scientist to trace a line across the feature in the context image. Once created, this line is propagated automatically across all element maps and intersections, solidifying the exact location of the same feature across all maps.

My Role

The primary transformation in this case study occurred when my team was able to understand the underlying scientific need or inquiry. The science team, partially because of their expert status, was unable to see that their thinking had been sculpted by the contingencies of their ecosystem of non-specialized tools. In identifying and helping the team to articulate precisely the line of inquiry that they wished to open, we were able to reveal a deeper need to fulfill through our visualization. In this case, the final form of the visualization is less important in and of itself, but reveals a novel form of inquiry, one which allows the scientists to focus on comprehending the sample as opposed to decoding imagery.