Between the Devil and the Deep Blue Sea: The Consequences of Small Failures in Learning
Julia Coffman, Center for Evaluation Innovation, May 2018
It was the worst U.S. maritime disaster in three decades, involving an experienced captain with a reputation for safety. What went wrong and what can this teach us about the often-overlooked factors that affect our learning and decisions?
I was flying across the country to attend the annual Grantmakers for Effective Organizations conference where we were doing a session on embedding learning into the way we work. Settled in for the 6-hour flight, I pulled out my favorite traveling companion — Vanity Fair. In it was an article by William Langewiesche titled The Last Words on the Bridge.
The article and the story transfixed me. It began:
“In the darkness before dawn on Thursday, October 1, 2015, an American merchant captain named Michael Davidson sailed a 790-foot U.S.-flagged cargo ship, El Faro, into the eye wall of a Category 3 hurricane on the exposed windward side of the Bahama Islands. El Faro means ‘the lighthouse’ in Spanish. The hurricane, named Joaquin, was one of the heaviest ever to hit the Bahamas. It overwhelmed and sank the ship. Davidson and the 32 others aboard drowned. They had been headed from Jacksonville, Florida, on a weekly run to San Juan, Puerto Rico, carrying 391 containers and 294 trailers and cars. The ship was 430 miles southeast of Miami in deep water when it went down. Davidson was 53 and known as a stickler for safety.”
What followed was an account of the conversation that occurred on the ship’s bridge during the last 26 hours before the ship went down.
The article offers a journalistic version of an after-action review, based primarily on the ship’s digital voyage data recorder, recovered 15,400 feet deep (over 3 miles) and only 2.5 inches long. Although the recording’s quality was poor, a technical team put together a 496-page transcript that was “a remarkable document — an unadorned record of nothing more than the sounds on the bridge.” The analysis was also based on separate Coast Guard and National Transportation Safety Board investigations, and the extensive documentation associated with all U.S.-flagged ships.
After El Faro set sail, its captain faced a complicated problem in which the solution — safely navigating through the oncoming hurricane — required both analysis and expertise. There were right answers and known solutions, however. In fact, other less risky routes had been mapped around the hurricane; El Faro’s captain had just chosen not to take them.
The bridge transcript makes it possible to know a good amount about what went wrong. We know the ship’s weather data was not as timely or as accurate as it could have been. Still, the conversation on the bridge suggested the crew knew they were headed toward the hurricane’s eye in enough time to avoid the worst of it. It was not just a failure of data.
The overall conclusion was that El Faro’s sinking resulted from a series of “system accidents…a cascade of small errors, failures, and coincidences” that affected how information was received, analyzed, and then used or ignored. El Faro can teach us a lot about how small accidents in learning can have major consequences.
Learning is About the Way We Work
Anyone who reads this is unlikely to captain a large cargo ship. But those of us who work in philanthropy do help to steer large strategies that can have real consequences if errors beset our learning and decisions. Like those on El Faro, many of these accidents are hard to detect and visible only in hindsight unless we pay attention.
The good news is that philanthropy now is paying much more attention to learning. Foundations are recognizing that learning is “real work” and must be integrated into their practices rather than treated as an optional add-on. Important infrastructure actors like Grantmakers for Effective Organizations, Center for Effective Philanthropy, Stanford Social Innovation Review, and The Foundation Review all are pressing this conversation in productive ways, highlighting the importance of paying attention to critical factors like organizational culture and feedback loops in our learning and decision making.
But in their efforts to support better learning, foundations are getting too tactical too soon. Many treat learning primarily as a supply-side problem — if we just provide the right data or information, then people will learn. Others look for quick-fix solutions, sometimes adopting private-sector knowledge management practices, such as dashboards or other tools, that are not always a good fit for philanthropy. The longer-term nature of many philanthropic missions and strategies — usually focused on the business of deep social and environmental impact — makes these kinds of tools less meaningful for guiding strategic decisions.
On the opposite end of the quick-fix equation, others are attempting to solve the learning challenge by designing comprehensive monitoring, evaluation, and learning “systems” out of whole cloth. After furious planning and sketching, they roll out new dashboards; theories of change and strategy templates; evaluation frameworks, guidelines, and expectations for program areas; grant reporting requirements; and learning agendas. Staff can feel crushed under the weight of it all and come to view both evaluative work and learning as a bureaucratic exercise — another set of hoops to jump through before they can get to the real work. Once that happens, the evaluative tools and templates have lost their real power, which is to help strengthen the way we think together about what we are trying to accomplish, and to improve our approach for getting there.
Rigorous learning is not simply a technical problem solved by having the right tool, the right template, or even the right data. It is a practice, a way of thinking and working, a set of capacities and habits.
Learning habits include, for example:[1]
· Making our thinking visible, identifying the hypotheses and assumptions that undergird our thinking and pinpoint about what we need to learn
· Asking powerful and meaningful questions relevant to how we do our work
· Having routines for gathering high-quality data and combating our biases
· Paying attention to causal inferences and the relationship between our actions and their outcomes
· Integrating reflective analysis and adaptive decisions into our day-to-day work.
Overlooked Factors That Affect Learning
Building and reinforcing good learning habits requires us to assess what is getting in the way of, or reinforcing, our effective use of information. We need to examine the personal, social, and institutional or structural factors that affect our ability to learn.
While this list is not exhaustive, six of these factors, some of which represent unexpected influences on learning, include the following.[2]
1. Personal motivation
The motivation and attitudes we bring to the workplace. Social stressors trigger a workplace “threat response” and manifest as an unwillingness to engage in open and candid dialogue. They include insecurity about status; anxiety about uncertainty; fear of losing autonomy; not feeling part of the same social group; doubt that decisions are fair.
2. Role identity
How we think about our roles and what it means to be good at them. We have “scripts” in our heads about what kinds of behaviors our roles call for and we behave in ways that fit with our identity’s stereotypes. Learning and adaptation need to be part of our script.
3. Relationships and connections
The kinds of relationships we have and with whom. How we relate on teams, across teams, and with partners (e.g., with deference, trust, humility, etc.) can affect our learning habits both positively and negatively.
4. Power and norms
Who controls and influences how information is accessed and gets interpreted. Our motivation and ability to learn is affected by who has formal control or informal influence over what the agenda is; what options are on the table; what data has value or is valid; how information is interpreted; and who blocks or enables information flow.
5. Rewards and incentives
The kinds of behaviors (or choices) that get rewarded or incentivized and how. Our learning habits need to be reinforced with formal and material rewards (e.g., performance reviews, promotions, grantmaking resources), or informal and psychological rewards (e.g., engagement and praise versus scrutiny and criticism).
6. Workspace
The space our teams, organizations, or networks occupy and how we connect. Our opportunities for interaction — physical and virtual — are key to learning. Who do we encounter? How and where do we connect? What mood do we experience when we do? How long does it take information to reach us?
Small Failures Can Sink Ships
All six of these factors were at play in the El Faro disaster.
· Personal motivation: Five years before El Faro sank, the captain had been fired from a position with another company for re-routing a ship at great expense when he learned of a problem with the steering gear. As a result, he had to sign on as a lowly third mate with his new company and work his way up again. This may have led to a reluctance to respond to safety concerns out of fear over another potential loss of autonomy and status. The captain didn’t want to take a different route that added extra miles, time, and cost.
· Role identity: The captain is the top person on the ship, literally at the helm. He or she is responsible for the ship’s seaworthiness, safety, security, navigation, cargo operations, and crew management. Being good at this role (running a tight ship) and maintaining the crew’s respect typically means displaying confidence, composure, and self-assuredness, along with some saltiness. This script likely factored into the captain’s ultimately overconfident decision to stick with a doomed route. That script also showed up in the captain’s demeanor during the final hours. Responding to the crew’s uncertainty and fear, the captain downplayed it. Even when waves were hitting at 13-second intervals, he jeered, “There’s nothing bad about this ride…I was sleepin’ like a baby.”
· Relationships and connections: Several crew members expressed doubts about the captain’s decisions and choice of route. While there were moments of “rebelliousness,” traditional maritime norms led the crew to be deferential and to reveal those doubts primarily to one another and not in front of the captain. As a result, the crew failed to challenge the captain’s thinking more rigorously. Challenges were expressed as passive-aggressive jokes rather than direct and data-driven questions. This dynamic was aggravated as “the watch” changed every several hours and crew members rotated on and off the bridge. Their collective power to both process information and challenge authority was diminished.
· Power and norms: In addition to the issues of decision-making authority associated with the formal hierarchy of rankings (captain, mates, helmsmen, etc.), another subtler power issue came into play in the captain’s preference for one type of weather data over another. El Faro had access to two sources of hurricane data. One came via satellite receiver in text form and required someone to plot it on a map. The other came by subscription service in the form of colorful weather maps. While the latter was based on data that was at least 12 hours old by the time El Faro downloaded it, evidence suggests the captain showed a preference for this less accurate but more visually appealing format. [3]
· Rewards and incentives: El Faro and its sister ship were scheduled to be replaced by two brand new ships in the near future. The captain of El Faro had been denied a position as head of the first new ship, but he’d recently had a good performance review and was hoping to be named captain of the second ship. This may have produced an incentive to stay on time and on the original course.
· Workspace: In its final day, El Faro’s bridge was a physically and mentally taxing place. The ship was rolling wildly and eventually started listing severely; visibility was dismal to nonexistent; huge waves were hitting at short intervals; alarms were sounding constantly. This chaos was not conducive to rational judgment and discursive decision making.
In addition, the technology that connected the ship to good information was lacking. The anemometer for measuring the speed of wind was broken. The ship lost its television and internet reception. There was a time lag in the weather forecast data. El Faro did not subscribe to routing guidance that could have come with its hurricane forecasts. There was a glitch in the weather data software. All failures that limited how the crew accessed and processed information.
Learning by Force of Habit
Each of these small errors, failures, and coincidences on El Faro mattered. “Absent any one of them, and the disaster would not have occurred…This is the stuff of tragedy that can never be completely explained.”
The National Transportation Safety Board ultimately found the probable cause of El Faro’s sinking was “the captain’s insufficient action to avoid Hurricane Joaquin, his failure to use the most current weather information, and his late decision to muster the crew.” The Coast Guard’s Marine Board of Investigation also found the disaster’s primary cause was the captain’s decision to sail too close to the hurricane. Had he survived, he would have been charged with negligence.
Especially with problems and solutions that can be highly affected by context (like successfully navigating a ship or steering a philanthropic strategy), we must attend to the tangible and intangible factors that can surreptitiously derail our learning and decisions. Even a person’s work history, the tacit norms that govern our interactions, or the appeal of some types of data over others, can be consequential.
If we really are committed to learning then we have to attend to the deeper personal, social, and institutional or structural variables that can impact robust learning habits. Our results depend on it.
Julia Coffman is founder and director of the Center for Evaluation Innovation and co-director of the Evaluation Roundtable. Twitter: @Eval_Innovation.
Footnotes
[1] These ideas are drawn from a variety of sources, including our own observations, conversations with participants in the Evaluation Roundtable, and the collective insights of the Emergent Learning Community of Practice hosted by Fourth Quadrant Partners.
[2] Factors are informed by research from multiple social science disciplines, including social psychology, sociology, education, industrial/organizational psychology, and organizational development. It also is informed by grounded research that others have done on learning in philanthropy, including Grantmakers for Effective Organizations.
[3] The captain likely did not receive formal training in the subscription service system and did know about the time lag in the data.
