Measuring Up: How to Avoid Mugging Pascal in Safety & Risk
One of the pleasures of living away from the CBD is the larger block sizes for the family & pets to run amok on and never ending need to mow the lawn (can’t see myself installing the imitation stuff just yet). The drawback however is the long commute into work which currently sits for me at about an hour each way. Podcasts have been my saviour and whilst not great keeping the brain in low gear to focus solely on the road conditions at hand they do make the process a little more enjoyable, and give me a large amount of cannon fodder for wide ranging discussions with others for debate.
Over the last week I listened to the Tim Ferris Podcast which in this episode included an interview with Will MacAskill (@willmacaskill). Will is an Associate Professor in Philosophy at Lincoln College, Oxford and the podcast raised a number of topics which are applicable to safety and risk that piqued my interest and are worthy of rational discussion.
Firstly the podcast contained mention of metrics and measurement of charitable benefits being delivered by charities in both a financial and non financial realm. In particular there has been some fantastic work completed by organisations such as GiveWell, who is a nonprofit dedicated to finding outstanding giving opportunities and publishing the full details of their analysis to help donors decide where to give. Unlike charity evaluators that focus solely on financials, assessing administrative or fundraising costs, they conduct in-depth researchaiming to determine how much good a given program accomplishes (in terms of lives saved, lives improved, etc.) per dollar spent. Rather than try to rate as many charities as possible, they focus on the few charities that stand out most (by theircriteria) in order to find and confidently recommend the best giving opportunities possible (their list of top charities).
My first thought was wow, isn’t this absolutely fantastic and my second thought was this must be an absolute momentous task in which to do this with any empirical accuracy (they do list their limitations on their website and potential margins of error, hat tip for transparency GiveWell). I would not have thought the above was possible considering the various ethical dilemmas it poses and the criteria and formulas in which they would need to conceptualize and implement to get to their metric based research. At their request they even get reviewed by non-associated 3rd party individuals.
So herein lies my cross pollination in regards to safety & risk, in broad terms I would like to examine the underpinning assumptions below;
- How do we know what good we are doing when we invest resources (money) or time (labour) in our various pursuits at work in relation to lowering the risk profile for either the short term or sustainably over the longer term?
- Is what we are measuring a true indicator of risk and is our baseline accurate to measure against if we identify any deltas to the baseline?
- How do we know what the return on investment is when we compare two activities ie one hour coaching a front line supervisor or one hour conducting a systems audit?
- Which activities gives us an outcome which lowers the risk profile the most based upon all the variables inferred by the range of activities we can complete and what process should we use to measure this?
If we can’t identify the non neutral effects of our efforts as a function (cognisant that some actions that H&S/ R introduce may actually increase risk — scary thought but even doctors harm patients unintentionally) in relation to lowering the risk profile are we just mugging Pascal?
The story goes something like this regarding Pascal’s Mugging; Blaise Pascal is accosted by a mugger who has forgotten his weapon. However, the mugger proposes a deal: the philosopher gives him his wallet, and in exchange the mugger will return twice the amount of money tomorrow. Pascal declines, pointing out that it is unlikely the deal will be honoured. The mugger then continues naming higher rewards, pointing out that even if it is just one chance in 1000 that he will be honourable, it would make sense for Pascal to make a deal for a 2000 times return. Pascal responds that the probability for that high return is even lower than one in 1000. The mugger argues back that for any low probability of being able to pay back a large amount of money (or pure utility) there exists a finite amount that makes it rational to take the bet — and given human fallibility and philosophical scepticism a rational person must admit there is at least some non-zero chance that such a deal would be possible.
My question and consideration based on the inference above is more often than not are the activities we complete as H&S personnel (the mugger) accepted by operational management (Pascal) due to an inferred benefit which may or may not occur and can’t be quantified by ourselves (even if they have a positive effect)?
I don’t think we as a profession are alone in the above quandary as their are many functions which complete activities towards a strategic objective that have minimal data (except in a theoretical capacity) behind the inferences they have made nor the activities they complete. HR is another such function which is attempting to disrupt itself and is increasingly with the help of big data identifying the drivers of value and those activities which improve outcomes for operational management. A specific example would be the current abandonment of performance evaluations in favour of alternatives after in house research conducted by the likes of Accenture and others. Another such example would beKPMG dumping employee engagement surveys
I feel the majority of us have a long journey ahead with our organisations and as a profession until the risk and safety function is at a position where we have a robust process which substantiates the impact of our activities and we shift from being just a cost centre. I strongly feel rather than wait for other functions to demand the same rigour that many of them have had for years, we should be welcoming and participating in our own disruption to ensure we provide evidence based support to operational management. Because if we don’t or can’t do it, others will.
So where to start? What service / tools are out there to which can measure risk or provide a framework to measure resource inputs and risk impact outputs? Notwithstanding any process or system is subject to the ‘garbage in, garbage out’ principle and this requires organisations to have a firm grasp of what their risk appetite consists of and how they measure this prior to engaging with any platform, services provider or developing in house capabilities.
The traditional problem-solving methods taught serve us well for some of the everyday challenges, but they tend to be ineffective with “Wicked problems”involving a high degree of uncertainty. Why? Because, more often than not, these tools are based on a flawed model of human behavior. And that flawed model is the invisible scaffolding that supports surveys, focus groups, R&D, and much of the long held inferences by the H&S function.
Prior to any change management activity being undertaken I think it’s most useful to examine the underpinning assumptions which we have based our current management systems on through applying a sensemaking process. Even if we acknowledge that safety is a wicked problem we can identify areas which we can measure and identify if strong correlations are present.
A five step sensemaking process can assist (taken from An Anthropologist Walks into a Bar…)
Reframe the Problem
The goal is to understand the complex, subtle, often unconscious ways in which people interact with their surroundings.
Collect the Data
Because data collection in the sensemaking process is explicitly designed to challenge assumptions, it differs fundamentally from conventional analytics and research approaches. Rather than administer a hypothesis-driven survey, run big numbers, or conduct carefully scripted focus groups, researchers engage in the lives of their subjects. Crucially, they approach the research without hypotheses, gathering large quantities of information in an open-ended way, with no preconceptions about what they will find.
Look for Patterns
Without analysis, of course, data collection is just plain reporting. Analysis allows us to connect the dots and reveal patterns.The key to uncovering patterns is to find root causes, the fundamental explanations between two or multiple variables
Create the Key Insights
Returning to the data with a new lens, a team can ask, “What are we missing?” It can look for gaps between the industry’s assumptions and employees experiences. Themes begin to emerge. Just as individuals can suffer from confirmation bias (a reflexive seeking of only information that supports an existing position), so can entire organizations.
Build the Business Impact
Insights must be translated into initiatives. These then implemented, tracked and reviewed against the initial assumptions to validate the correlation. Only then can we be sure that we have made a temporary or sustainable impact in reducing or managing risk successfully to the degree identified
The innovation in sensemaking is less about the research technique than about the human-sciences-based analysis. Sensemaking reveals answers that conventional tools can’t, and it enables leaders to think creatively about what services they are really providing and if they add value.
“Better is possible. It does not take genius. It takes diligence. It takes moral clarity. It takes ingenuity. And above all, it takes a willingness to try.” ― Atul Gawande, Better: A Surgeon’s Notes on Performance
So where are the industry leaders which are attempting to push the curve forward? See below for some which spring to mind, that I have had the pleasure to come across but not limited to;
- MiProfile a tool is a surveying learning event which stimulates intense conversations about culture and at the same time measures culture in an organisation.
- “Let’s move away from the bureaucracy surrounding safety and move the psychology of safety perception away from an obsession with numbers. As we explored earlier: safety performance gets confused with accident rates. If we feel compelled to have to measure something to alleviate our concern that safety is continually improving, why don’t we make it something else? What would that target be, or perhaps more pertinently, would there be one?”
- Organisations that seek to embed sustainability and climate change into core business activities have an advantage with real benefits. We help organisations identify, develop and deliver strategies that address risk and opportunity while measuring their effectiveness.
Periscope (Electronic Management Solutions Framework)
- Periscope software is based on a flexible data capture system that can measure all types of enterprise data. You can analyse and monitor your strategic planning, track your goal setting, even evaluate your adherence to core values. Our comprehensive, customisable, and user-friendly planning capabilities let you grow your business in the way you want — simply
The Moment Of Clarity — ReD Associates (Framework — Sensemaking Process)
- In The Moment of Clarity, Christian Madsbjerg and Mikkel Rasmussen examine the business world’s assumptions about human behavior and show how these assumptions can lead businesses off track. But the authors chart a way forward. Using theories and tools from the human sciences — anthropology, sociology, philosophy, and psychology
- The Centre for Safety aims to be a vital link between research and industry. We create and support opportunities for researchers to engage with industry, while also providing industry with access to the wealth of knowledge and expertise that lies within universities
- When considering the role of board members and senior executives it is necessary to consider the corporate governance framework in which they operate. The concept of safety governance is designed to ensure that boards have the tools, knowledge and structures in place to maximise the safety performance of the organisations they govern and lead, beyond mere compliance with relevant safety legislation
- “The first full year the IBM predictive analytics solution was used in Tennessee was the second lowest traffic fatality year since 1963,” reports Colonel Trott. “The figures speak for themselves.”
My thoughts are that all H&S personnel need to be part of the disruption find something new to try, something to push a frontier. Count how often it succeeds and how often it doesn’t, but count something. Write about it, be brave, don’t worry about the critics, be the man in the arena. Ask a colleague what they think about it. See if you can keep the conversation going. Talk to a researcher (Laura Fruhen if you don’t know one). Have an integral perspective, be a dualist, transcend but appreciate all the H&S worldviews which have come before and added value (BBS, Zero Harm etc) but don’t be that person at the table with no data to back up your assertions, activities or initiatives. Understand the risk profile your business operates in and methodically attempt to reduce it through continuous improvement with the practicable means available to you and your circle of influence. Know what your drivers of value are and pursue them ruthlessly to make a difference.
This is my Why, I hope you share it and above all, be useful.
If you are working for an organization which is ahead of the point on the curve mentioned above I would love to hear via comments (or twitter @edawneedham) on this post how you overcame this to add to the list above and what tools or process you have used which others can leverage to make a difference
Originally published at www.linkedin.com.