An introduction to the Scientific Abilities
When diamonds are your worst enemy
The first class I taught at Stanford was a first-year seminar on Geoforensics, called CSI: Stanford. Geoforensics is the idea that the inherent natural variability in Earth materials, from rainwater to rocks, provides unique fingerprints that can be used to track the movement of not only objects, but also people and animals (John McPhee wrote a great story for the New Yorker). Every year I would invent a case and the students would use different tools to analyze the evidence. One year I partnered with a colleague who was teaching another first-year seminar called “Diamonds” to see if we could use some cutting-edge instruments to analyze diamonds and eventually determine ones that might have been illicitly sourced. “Can we fingerprint diamonds to determine their geographic origin?” Our hypothesis was that each diamond locality should have a unique chemical composition based on when and where it formed.
We were able to get a number of rough diamonds with known sources from colleagues, contacts in industry and our own Stanford mineral collection, to analyze. As the data started to come in, there were many things we couldn’t explain, including both fascinating chemical variations in single diamonds and nanometer-scale visual patterns. We scoured the small amount of peer-reviewed literature on diamonds. Nothing was helpful. We reached out to colleagues at MIT, Caltech and the Smithsonian to help us understand what we were seeing. They had no idea either.
Given the brevity of the quarter, we had to press ahead. To me, it was infinitely exciting that we had discovered all these patterns, as well as the lack of expected patterns, that we couldn’t explain. However, as I began to present the data to our students so they could attack the hypothesis I felt a growing sense of panic in the room.
“What do you mean ‘you don’t know’?” asked one brave student, skeptically. The rest mostly stared back, uncharacteristically quiet.
For many of the students in the class this may have been the first time they truly pushed up against the boundary of knowledge. They were in what learning theorists would call the “Red Zone of Panic”. Why was it so startling? Many of you may confront similar boundaries of knowledge during your summer research. This, after all, is the central goal of research. You will have data you and your mentors can’t understand, models that predict unexpected things, and people that answer questions in unanticipated ways. It may feel scary and foreign to you, and at times frustrating. At least, this is how it always feels to me, even after 20 years as a scientist.
Why is confronting the boundary of knowledge so scary? I think there are two related reasons. The first is our natural response to new information and the second is an over emphasis on science as a set of discrete steps, rather than as a collection of abilities we weave together. If we understand these abilities, we can use them to advance beyond the Red Zone of Panic into what we call the Yellow Zone of Learning.
Let’s use an example. Many of you have probably had a professor that was so engaging in lecture and so clear in their presentation of new knowledge that you were convinced you were taking it all in. Fast forward to the exam, and all of a sudden you realize you didn’t know the material as well as you thought. Listening to an engaging lecture feels familiar, comfortable. We will call this the Green Zone of Comfort. It turns out we probably don’t learn very much if we are always in the Green Zone.
Let’s assume the same engaging lecturer suddenly stops and gives you a problem to solve on your own. At first you probably feel a bit of Red Zone panic. What do you do? Most likely you desperately look back through your notes. You are pulling out the key themes that have been discussed — what are the main principles of the concept? You are exercising your synthesis abilities. You find an example and analyze it. It is not exactly like your problem, it is far more abstract, but it shows how the principles can be applied to a problem, so you start to map the example on to yours. This is an ability called moving between concrete and abstract. Finally, you are starting to see how this principle works and how you might apply it to a new problem. This is the Yellow Zone of Learning. It feels hard and a little uncomfortable, but you are actually learning. As you work the problem, you start to feel more excited, maybe even exhilarated.
Next, your professor asks you to compare notes with your neighbor. You are pretty confident you nailed it, but you both have different answers. As you dig in, you realize you made an assumption that led to a different conclusion. As your partner explains their thinking, you realize that your assumption might have been questionable. Here, you are learning from others. It still feels uncomfortable, especially admitting you made a mistake, but it is still solidly Yellow Zone. During the class-wide reflection, you realize that 50% of the class made the same mistake and now you understand why. Reflection is also part of how we learn.
All of us as learners have also spent time in the Red Zone of Panic. It can feel stressful, frustrating, tense and full of ambiguity. Think about a time you were in the Red Zone. How did you navigate? Did you talk to classmates? Go to office hours? Check out another textbook? Google exhaustively? All of the above? Think about the mechanics of how you transitioned from Red to Yellow. Most likely you were using a few abilities from above, and another, called navigating ambiguity.
Ultimately, in sharing the complex and unexplained diamond results with my students I had inadvertently put them in the Red Zone, but why?
When the scientific method is not enough
To be sure, there were lots of things to induce panic: first time plotting large datasets in excel, lots of chemical elements, and strange sounding parameters like “d-spacing”. However, I think at the heart of it was a recognition that the canonical arc of the scientific method had collapsed. We asked a question, collected data, and now we should be able to analyze the data and answer the question. After all, this is what it looks like in the scientific literature? Don’t be deceived. Ask any scientist to chart their journey that led to their most recent publications and you will find it looks more like a hairball than a nice linear journey though the scientific method. In reality, scientists rely on a set of abilities, including those in italics above, that enable them to define and then solve important problems.
How can the abilities help us to become better learners?
We tend to like recipes, so when we feel like the scientific method is failing us, it’s easy to end up in the Red Zone. As researchers, we need to find our way into the Yellow Zone but also be weary of the comforting allure of the Green Zone. The Yellow Zone is where we are learning, which is, after all, the point of doing research.
Here, I want to want to introduce another framework that we will use throughout the quarter. Instead of focusing on steps (i.e., the linear scientific method), we are going to focus on the abilities you need to move fluidly through your own journey of learning and, by extension, knowledge creation.
First, we’re going to simplify the scientific method into a Venn diagram to the left, where new knowledge production arises from the intersection of three processes: domain expertise, problem definition and problem solving. If you take one of these away, it is very difficult to produce new knowledge. For example, if you’re still learning about a particular discipline, it can be hard to define a good problem to solve. Your mentors probably used their domain expertise to help you identify the research question in your proposal, but that doesn’t mean they got it right. In fact, you may have to re-define your research problem in the course of your summer experience.
Looking at only the processes, being a scientist still feels a bit intangible. How does one become good at problem definition?
As the second part of our process, we are going to assume that certain abilities allow us to do integrate these three processes effectively. It turns out some colleagues at the d.school, who specialize in teaching people to be creative problem solvers, have thought a lot about this. Carissa Carter, the Director of Teaching and Learning, and one of the most brilliant thinkers I know (incidentally, her original field was geomorphology), defined a set of abilities [you can read her design version and original manifesto here]. Some of the abilities were highlighted above but the rest are shown in the figure below followed by brief descriptions. Here, I have repurposed the design abilities to help us see how we can use them in a scientific setting. Some of you will already be really good at some of these, others might be abilities you want to pull out and focus on as part of your summer learning objectives.
Synthesize Information. This is the ability to make sense of information from diverse sources, including peer-reviewed journals, and find insight and opportunity within.
Navigate Ambiguity. This is the ability to recognize and stew in the discomfort of not knowing which direction to go with a research thread, and then come up with tactics to emerge out of it when ready.
Communicate Deliberately. This is the ability to form, capture, and communicate stories, ideas, observations, concepts, reflections, and learnings to the appropriate audiences.
Experiment Rapidly. This ability is about being able to quickly generate ideas, whether written, drawn, or built.
Move Between Concrete and Abstract. This ability contains skills around understanding stakeholders as well as zooming and expanding on product features.
Build and Craft Intentionally. This ability is about thoughtful construction and showing work at the most appropriate level of resolution for the audience and feedback desired.
Design your Research. This meta ability is about recognizing the people, tools, techniques, and processes to you need to tackle your question.
Learn from Others (People and Contexts). This ability includes the skills of empathizing with different people, as well as testing new ideas with them.
More examples of how scientists use abilities and why we should teach them can be also found here: The Fog of the PhD.
How do we use abilities synergistically?
Let’s dig into problem definition. Hang around Stanford long enough and you will be asked: “What problem are you trying to solve?” Too often this is done in a condescending way. This makes me mad. Problem definition is hard and it requires one to use several, if not all, of the abilities above. As an example, let’s assume you are interested in water. That is a big topic. Maybe you have taken some classes on water policy or maybe you just love rivers so you decide to read some journal articles on both. On one hand, you learn how hard it is for managers to predict future water availability, on the other hand, you learn about the negative ecological consequences of dams. You have just done some excellent synthesis but you still don’t have a clear idea of which one you should focus on — you are navigating ambiguity. This might feel like classic Red Zone territory. Sometimes it can help to shift your angle. You have been thinking at a very abstract level but what happens if you start to get more concrete? You can do this by imposing constraints: What if I can only work in California? What if I can only work with Stanford faculty? What if we can’t build any more dams? What if I can only use a computer?
Sometimes you need to go other the other way towards more abstract, this scale shift is called moving between concrete and abstract and can help you see a problem in a new way.
Digging into California issues, you talk to your Earth 1 TA (i.e., learning from others) and they tell you about an idea, called managed aquifer recharge, where water managers want to pump excess river water into the ground for storage. A light bulb blinks and suddenly your interests and domain knowledge are colliding. You are solidly in the yellow zone now and feeling excited about some research questions:
“Can deployment of managed aquifer recharge in California improve predictions of water availability?”
“Does diversion of rivers flows for managed aquifer recharge impair downstream ecological conditions?”
Both are potentially excellent research questions but the first one might be addressed using computer models, your particular passion. It harder to see how you would answer the second one, so maybe we can cross that off for now. By bringing in the constraints, whether they are your personal interests (love for rivers), your geographic restrictions (California), or different skillsets (computer programming and knowledge of water policy), and using them to scope your question, you are designing your research by being intentional about selecting a problem you have the resources or tools to solve.
This is an introduction to the scientific abilities and how we think you might use them in your summer research and beyond. Most importantly, they are the abilities we need to be effective good problem solvers, and the world desperately needs more of us. Your summer is your time to focus on the abilities that you want to magnify we hope that by the end of the quarter you will have chosen one or two to focus on.
Most importantly, we hope you will recognize the different learning zones, when you are in them, and how the abilities can help you navigate between them to optimize your time in the Yellow Zone.
What happened to the CSI:Stanford students?
We spent some time together building excel templates to easily visualize the data, starting with a small fraction of the data. I introduced them to exploratory data analysis (EDA), which is a fancy name for plotting everything against everything. Investment in learning a tool you need (excel, python) is one way to move beyond the Red Zone. We also re-defined the problem to focus on the trends (or lack thereof). One group of students was able to show that synthetic diamonds are less variable than natural diamonds in their chemical composition, while another group found that South African diamonds had consistently higher potassium concentrations. We also came up with a set of recommendations for additional research that would be needed to advance our ability to track illicit diamonds using the methods we tried. Ultimately, the project added nuance to the central learning objective of the course — Earth materials are complex and variable at all scales. Sometimes that variability is systemic and allows us to use it to our advantage, whereas other times it can defy understanding, at least for a while. However, it is that sheer defiance of the natural world to be easily understood that can make science such a compelling adventure.
“That thing the nature of which is totally unknown to you is usually what you need to find, and finding it is a matter of getting lost” — Rebecca Solnit “A Field Guide to Getting Lost”