Visual Thinking & Graphic Discoveries
by Michael Friendly & Howard Wainer
This article started out as an addendum to a chapter in our book, Data Visualization: A History of Visual Thinking and Graphic Communication (Friendly & Wainer, 2020). In this we claimed that much of the history of data visualization could be seen as combination of three forces: (1) important scientific problems of the day, (2) a developing abundance of data, and (3) the cognitive ability of some heroes in this history to conceive solutions to problems by visual imagination.
In the book and what follows we make frequent reference to cognitive aspects of the visual understanding of phenomena and their expression in graphic displays: “inner vision”, “graphic communication”, “visual insight” are some of the terms we use. An early metaphor for this and an early title for our book was “A gleam in the mind’s eye.” We give some additional explanations and examples here. We also want to place this topic in a wider framework.
Visual communication
Commonplace examples of visual communication abound in everyday life. One of us was totally baffled by the verbal instructions for installing a child car safety seat[1] and even more so by the instructions for setting time zones on a fancy digital watch (Figure 1).
[1] A 2003 study by Wegner and Girasek in the journal Pediatrics of the readability of such instructions was motivated by a report that over 80% were improperly installed.
In contrast, the visual instructions that accompany IKEA furniture products are a model of clarity (Figure 2). You have only to follow the pictured steps carefully, and pay attention to the small blown-up detail diagrams to make sure you understand which side is up or on the left, but success is usually guaranteed if you do.
Similar clarity is present in effective presentation graphs of data. In 2015 Tynan DeBold & Dov Friedman in the Wall Street Journal tried to show the effect of the introduction of vaccination programs in the US states on disease incidence, using color-coded heat maps for a variety of diseases.[2] Their graph for measles (Figure 3) shows dramatically that measles outbreaks were reduced by 90% following the introduction of the vaccine in 1962. Sadly, vaccination rates declined after 2014 and measles re-emerged in those areas with lower immunization rates. Effective graphs can cure ignorance, but not stupidity.
[2] http://graphics.wsj.com/infectious-diseases-and-vaccines/
The message in such effective graphs hits you between the eyes. Joe Berkson, a statistician at the Mayo Clinic, expressed this as the “Inter Ocular Trauma Test” — — for a graph so clear as to require little in the way of statistical significance tests or detailed explanation. This was probably not what Yogi Berra was thinking when he said, “You can see a lot by looking,” but it doesn’t miss the mark by much. These examples convey the result of clear visual communication, but not what led to it.
Mental imagery and visual imagination
Related terms are “mental imagery” and “visual imagination”. In literature, some great fiction writers, such as Charles Dickens, provide compelling examples of their ability to imagine scenes and translate these into words conveying the rich detail of setting, thoughts and emotions of the characters, so that the reader can see these too and connect with their own experience. Underlying this is the cognitive capability of mental imagery, a capacity to imagine scenes and memories in rich, vivid detail.
To test yourself, imagine a scene at a recent restaurant meal with family and friends. Close your eyes. How vividly can you see the table, the restaurant setting in your mind’s eye? What was your server wearing? Do you have images of the sequence of dishes? For some people, this is easy and natural; others can remember many details, but they can’t replay the scene in mental images.
The polymath Victorian scholar Sir Francis Galton was the first to study this topic scientifically as a mental ability that people possess to more or less degree (Galton, 1880a, 1880b). He stated this most clearly as:
“There are great differences in the power of forming pictures of objects in the mind’s eye; in other words, of visualizing them. In some people, the faculty of perceiving these images is so feeble that they hardly visualize at all …. Other persons perceive past scenes with a distinctiveness and appearance of reality that differ little from actual vision. Between these wide extremes I have met with a mass of intermediate cases extending in an unbroken series”. [Galton, 1880b, p. 312]
In his groundbreaking experiment, Galton devised a questionnaire that he distributed to 100 male acquaintances, either “distinguished in science or in other fields of intellectual work” (Galton, 1880a, p. 304) or others. The questionnaire begins by asking the subjects to “think of some definite object — suppose it is your breakfast-table as you sat down to it this morning — and consider carefully the picture that rises before your mind’s eye.” He asked them to record aspects of the quality of their mental image, e.g., “1. Illumination — Is the image dim or fairly clear? Is its brightness comparable to that of the actual scene?” Other questions were asked about the degree of definition of the image, color, the field of view, and so forth.
Although Galton himself was highly gifted in visual thinking, he concluded, surprisingly (among other things) that his sample of 19 Fellows of the Royal Statistical Society showed little inclination to rich visualization. In part, this reflected Galton’s view that there is a wide distribution in the ability or inclination to visual thinking that could be related to gender, class and age. But his explanation reflects a gender and class bias of the Victorian age and an egregious extrapolation beyond his data. Scientific men, he said, might confess to feeble powers of visualization because “an over-readiness to perceive clear mental pictures” is “antagonistic to … habits of highly generalized and abstract thought.” On the other hand, “women and intelligent children” have vivid mental imagery and can report this in detail.
Galton’s claims about differences in mental imagery among general groups (scientists vs. non-scientists; men vs. women) have largely been refuted by later research (Brewer & Schommer-Atkins, 2006), but he established the idea that subjective experience of mental imagery could be studied scientifically. Indeed, in a tongue-in-cheek reply to Gelman (2011), one of us proposed that the psychology of data visualization could be understood in terms of “graph people” vs. “table people” (Friendly & Kwan, 2011).
In extreme form, some people have been claimed to have eidetic or photographic memory: a brief glance at a page of text is sufficient to recite it, forward or even backward. Even more extreme is the case of a young man, called “S.”, reported by the Russian psychologist, Alexander Luria in The Mind of the Mnemonist (Luria, 1987). “S.”- was a reporter for a Moscow newspaper, who later became a professional mnemonist, demonstrating nearly limitless ability to remember random words, numbers, poems in a foreign language, almost anything the audience could supply. His secret was that he translated everything into rich, immutable sensory images containing tastes, smells and visual images,[3] something now called synesthesia.
[3] The story of the mnemonist told by Luria was recently revisited by Reed Johnson in New Yorker article (August 12, 2017), https://www.newyorker.com/books/page-turner/the-mystery-of-s-the-man-with-an-impossible-memory . The true story of S. turns out to be more complicated than told by Luria.
Galton’s work spurred many follow-up studies of individual differences in mental imagery. Luria’s account of an individual so possessed by eidetic imagery that he could barely function in normal life and only found work as a professional memory performer excited modern cognitive scientists to study this topic in far more detail.
With the advent of functional magnetic resonance imaging (fMRI) a new cohort in cognitive neuroscience has tried to tie these differences more directly to what could be observed in a human brain: what brain areas light up when presented with a verbal joke, an emotional art work (Picasso’s Guernica) or an emotional data visualization (Minard’s graphical portrayal of Napoleon’s failed 1815 campaign). A detailed history of the philosophical and empirical roots of what can be called “imagery science” is given by MacKisak et al. 2016.
Visually-based discoveries
In our book, there are many stories of the origins of data visualization we tell through this lens. Many are classic, but deserve re-telling from a modern perspective: Michael Florent van Langren and the first statistical graph, André-Michel Guerry and the birth of modern social science, John Snow on cholera and the origins of epidemiology, and, of course Florence Nightingale and the power of graphs for political persuasion.
There are many additional stories of visual thinking and graphic discovery that found no place in our current book, but are still useful. Some of these were related in an earlier account (Wainer, 2005). We’ve chosen a few less well-known orphans that express this theme in this essay. All of these are drawn from the Milestones Project (http://datavis.ca/milestones ), and the selection here relates to topics in physical science.
Benzene: Dreams, snakes and hexagons
Among the most dramatic (though perhaps apocryphal) example is the story of the discovery of the chemical structure of the organic compound benzene by the German chemist, August Kekulé around 1862. This story is told to almost every introductory chemistry student as an example of what they can achieve, by mastering the periodic table and methods of chemical analysis and composition. The chemical composition of a benzene molecule had been known for some time; the formula C6H6 says that it is composed of six carbon atoms and six of hydrogen, but no one understood a physical structure of the atoms to account for its observed properties.
The standard story (told by Kekulé himself, 25 years later) was that, dozing by the fireplace, he had a dream in which he saw these atoms dancing around. They began to form themselves into connected strings, and the strings formed the shape of a snake, biting its tail. The mystery was solved, with six carbon atoms connected in a hexagon, each connected to one hydrogen atom. In Kekulé’s words:
“One of the snakes had seized hold of its own tail, and the form whirled mockingly before my eyes. As if by a flash of lightning I awoke; and this time also I spent the rest of the night in working out the consequences of the hypothesis.”
But he also cautioned:
“Let us learn to dream, gentlemen, and then perhaps we shall learn the truth . . . but let us beware of publishing our dreams before they have been put to the proof by the waking understanding.”
Figure 4 is a common visual representation of Kekule’s dream. The six carbon atoms form a hexagon, and the connecting lines show an alternating pattern of single and double bonds. The hydrogen (H) atoms form the periphery. Altogether, the diagram is a perfect visual representation of a theory that accounts for the chemical properties of benzene. More importantly, with this insight, Kekule solved a more general puzzle: how to depict chemical structures based on the fundamental property of atomic valence. This is the power of visual thinking.
The representation of a snake or serpent biting its tail is called an ouroboros, something that goes back to ancient Greek and earlier magical thinking. In modern western tradition, it is directly traced to a drawing by Theodoros Pelecanos in a 1478 tract, Synosius on Alchemy. Is it possible that this image intruded in Kekule’s slumbers?
Before we leave this topic, there is another aspect of Kekulé’s dream worth noting: He could have just wrapped the carbon atoms in a circle, with similar properties, but some other visual or chemical insight caused him to chose a hexagon. We can’t speculate on this, but just note that hexagons provide one optimal geometric solution for packing regular shapes together with no wasted space. This is the reason that some modern algorithms for binning data in 2D space use hexagonal regions, and why many R packages use hex logos.
Mendeleev and the periodic table
Dimitri Mendeleev [1834–1907] is well known as the constructor of the periodic table of chemical elements, but the story of how he got there is worth recounting,[4] and appropriate in the 150th anniversary year of its discovery.
In 1867 he began writing a Principles of Chemistry book, and set out to try to organize the known chemical elements: most basic were hydrogen, oxygen, nitrogen, carbon; another class had been called halogens: chlorine, bromine, iodine; there were also alkali metals: lithium, sodium and potassium; other metals: copper, silver and gold were also well-known. Around this time, there were 63 known chemical elements. Most had been classified by atomic weight, starting with hydrogen, with weight 1.
He set out to organize and explain the elements. He began with what he called the “typical” elements: hydrogen, oxygen, nitrogen, and carbon. Those substances demonstrated a natural order for themselves. Next he included the halogens, which had low atomic weights, reacted easily with other elements, and were readily available in nature. He had begun by using atomic weights as a principle of organization, but these alone did not present a clear system.
At the time, elements were normally grouped in two ways: either by their atomic weight or by their common properties, such as whether they were metals or gases, and how they reacted with other elements. Mendeleev’s breakthrough was to see that these all could be combined in a single, unifying framework.
[4] Details of this story follow https://www.khanacademy.org/partner-content/big-history-project/stars-and-elements/knowing-stars-elements/a/dmitri-mendeleev and https://blog.oup.com/2012/08/how-exactly-did-mendeleev-discover-his-periodic-table-of-1869/
As this story is told, Mendeleev wrote all the known properties of the 63 elements on a set of cards. He carried these everywhere and sorted these in various ways, but was unable to see any global organizing principle that would account for all properties. It is also said that his inspiration came from the game of solitaire, where cards are arranged by suit (horizontally) and number (vertically) in the form of a table.
On February 17, 1869, right after breakfast, and with a train to catch later that morning, Mendeleev set to work organizing the elements with his cards. He carried on for three days and nights, forgetting the train and continually arranging and rearranging the cards in various sequences until he noticed some gaps in the order of atomic mass. He later recalled, “I saw in a dream, a table, where all the elements fell into place as required. Awakening, I immediately wrote it down on a piece of paper.” (Strathern, 2000) He named his discovery the “periodic table of the elements.”
After his dream, Mendeleev drew the table he had envisioned. After several incomplete drafts he arrived at Figure 5. The modern version of the periodic table is the transpose of Mendeleev’s: The six columns in his table are now called “periods,” and the rows indicate a “group,” or family of elements. What Mendeleev saw was that this table could be read by columns, to show some similar properties, and by rows to show other properties of the elements. The main ordering was by atomic weight, starting with the lightest element, H=1. Within periods and groups, elements shared different common properties of appearance and behavior: melting and boiling points (temperatures at which they change from a solid to a liquid, or a liquid to a gas); reactivity with other elements, and so forth.
Most tables today are just lifeless, desiccated collections of numbers, laid down in a form to facilitate only one task: lookup. How many measles cases were there in Iowa in 1940 (see Figure 3)? Modern versions of the periodic table (e.g., https://www.ptable.com/ )are dynamic and interactive and show a host of properties of chemical elements in ways that can be understood and explored.
The concept of a periodic table was recently adapted by Ralph Lenger and Martin J. Eppler (http://www.visual-literacy.org/periodic_table/periodic_table.html ) as an attempt to organize visualization methods in more general ways. Figure 6 on the web is interactive in a limited way: hovering the mouse over any cell shows an example and some other details. The methods are color-coded to show various categories of visualization. To what extent this is useful remains to be seen, but it attests to the utility of Mendeleev’s idea to organize distinct elements with different properties in a visual table.
Henry Mosely and the discovery of atomic number
The hallmark of good science is the discovery of laws which unify and simplify disparate findings and allow predictions of yet-unobserved events or phenomena. Mendeleev’s periodic table, for example, allowed him to predict the physical and chemical characteristics of Gallium (Ga) and Germanium (Ge) before they were discovered decades later.
Mendeleev’s table, however, arranged the elements only by a serial number, denoting an atom’s position in a list arranged by increasing atomic mass. This changed in 1913–14 when Henry Gwyn Jeffreys Moseley [1887–1915] investigated the characteristic frequencies of X-rays produced by bombarding each of the elements in turn by high energy electrons. He discovered, that if the serial numbers of the elements were plotted against the square root of frequencies in the X-ray spectra emitted by these elements, all the points fell almost neatly on a series of straight lines (Figure 7).
“Now if either the elements were not characterized by these integers, or any mistake had been made in the order chosen or in the number of places left for unknown elements, these regularities would at once disappear. We can therefore conclude from the evidence of the X-ray spectra alone, without using any theory of atomic structure, that these integers are really characteristic of the elements.”
This must mean that the atomic number is more than a serial number; that it has some physical basis. Moseley proposed that the atomic number was the number of electrons in the atom of the specific element, as opposed to atomic weight or mass. His use of a square-root scale was not accidental; it had been suggested by Niels Bohr’s theories regarding electrons in various shells.
Moseley’s graph represents an outstanding piece of numerical and graphical detective work. He noted that there were slight departures from linearity which he could not explain; nor could he explain the multiple lines at the top and bottom of the figure. The explanation came later with the discovery of the spin of the electron. Most physical scientists agree that Mosely would certainly have received the Nobel Prize, had he not died at age 27, felled by a Turkish bullet on the Gallipoli Peninsula on 10 August 1915.
Conclusion
Stories of scientific discovery are useful if they can shed some light on the questions: “What were they thinking?” “How did they come to understand things in this way?” In our book, we trace the origins of graphical methods in relation to scientific problems whose solutions were facilitated by visual thinking and seeing lawful regularity in otherwise chaotic numbers. John Snow’s discovery of cholera as a water-born disease is perhaps the most well-known example, giving rise first to dot-maps of disease incidence and later to sophisticated models and graphical methods that smooth the data to help explain the pattern.
In this article we try to take the idea of visual thinking further: First with a more general description of these ideas and history; then with three lesser-known examples from the physical sciences. None of these led to innovations of graphical methods, but they changed scientific understanding in their time. As we describe, they all relied on abilities to see things differently in the mind’s eye.
References
Brewer, W. F. & Schommer-Aikins, M. (2006). Scientists Are Not Deficient in Mental Imagery: Galton Revised. Review of General Psychology, 10, 130–146.
Friendly, M. & Kwan, E. (2011). Comment (Graph people versus Table people) Journal of Computational and Graphical Statistics, 20, 18–27.
Friendly, M. & Wainer, H. (2020). Data Visualization: A History of Visual Thinking and Graphic Communication. Harvard University Press, in press.
Galton, Francis (1880a). Statistics of mental imagery. Mind, 5, 301–318.
Galton, Francis (1880b). Mental imagery. Fortnightly Review, 28, 312–324.
Gelman, A. (2011). Why Tables Are Really Much Better Than Graphs
Journal of Computational and Graphical Statistics, 20, 3–7.
MacKisack, M.; Aldworth, S.; Macpherson, F.; Onians, J.; Winlove, C. & Zeman, A. On Picturing a Candle: The Prehistory of Imagery Science. Frontiers in Psychology, 2016, 7, 515
Moseley, Henry (1913). The high frequency spectra of the elements. Philosophical Magazine, 26, 1024–1034.
Strathern, Paul (2000). Mendeleyev’s Dream: The Quest for the Elements. New York: St. Martin’s Press.
Wainer, Howard (2005). Graphic Discovery: A Trout in the Milk and Other Visual Adventures. Princeton University Press.
