Part 1: Getting metrics right is all about ‘Who?’
Putting people front and center in metrics leads to better results from the ‘what,’ ‘how,’ and ‘why’
In the last few months, I’ve seen a lot of data and analytics thrown around — about COVID19, the economy, the future, etc. I suspect that ‘becoming more data-driven’ will be a popular strategy for surviving and recovering from the pandemic. That’s good, yet I worry about treating this as a silver bullet — plenty of data and analysis is pretty much useless garbage.
My work in data and analytics has spanned a wide variety of contexts. I was a nerd all of childhood, worked in a cancer research lab in college, graduated with a pre-med degree, studied social science in grad school, worked in a data-driven education culture at Teach for America, and saw a variety of business analytics in my years as a consultant. While I’ve learned a lot in all these experiences, I consistently find there is one key aspect of metrics that is often ignored or deprioritized.
Peter Drucker said “you can’t manage what you can’t measure.” I’ve sometimes adopted a more provocative version : “if it isn’t measured, it doesn’t exist” (pretty sure my friend Scott’s the first person I heard say this). Whether it’s Drucker’s phrase or mine, I’ve most often seen these statements used to emphasize the importance of metrics and data-driven processes. Perhaps some of these examples — beloved or hated — might be familiar to you:
- Large organizations develop complex dashboards and analytics engines for executives to pore over in guiding the enterprise
- Lean startups aim to pick a few key measures and treat them as the only things that ultimately drive the company
- Pharmaceutical companies fill ad space with impressive numbers about how their drugs can make you better, faster, stronger, happier, etc.
- Elite consulting firms offer complex frameworks (and project contracts with lots of zeros) to make sure you’re measuring the right things to keep up with your competition
- Six Sigma blackbelts focus on a few specific production metrics (with a library of mystical words, processes & tools, and magic tricks to drive out ‘waste’)
- Non-profits have glossy annual reports full of heart-strings stories and analysis of how your donation (with just a few zeros) can make an impact on an issue you care about
- Business school professors provide you with the latest research on how to make your company data-driven (and coaching contracts with, you guessed it, lots of zeros)
I’ve come to appreciate many good things about all of the above examples. At the same time, I’m often frustrated because I find that engineers, scientists, business people, and even often medical and social impact workers — who have been dominant groups in much of my education and career — often treat data and their analysis of it as THE thing. Here, I’m grateful to my training as a social worker and an educator for helping me express what’s behind this gut discomfort: at the core of any metrics that matter is not the data itself or analysis of it, but PEOPLE. It is the individuals and groups of PEOPLE impacted that ultimately give all data its meaning and value.
Because of this perspective, I work to put the question “Who?” front and center. In honing this habit, I’ve learned a variety of prepositions and phrases can be added. Related to metrics, I often ask:
- Who is (and isn’t) being measured? — this is the age old question about the nature of the statistical ‘sample.’ I found this recent article from Nature enlightening — key point: different groups of people have vastly different thoughts, feelings, behaviors, and patterns. It’s important to identify the group of people to include in your sample that will most help you reach your goals. If you’re launching a new eCommerce portal, when previous sales have all been local retail, all your current customers aren’t the right guide — focus instead on current customers that often shop online and/or people that represent new customers you want to bring in. If you’re launching a start-up aiming to sell a product mass market, don’t ask only highly educated, affluent friends for feedback — you also need feedback from people with a wide variety of educational and socioeconomic backgrounds, so your product is tailored to meet a diverse set of socioeconomic expectations.
- Who is (and isn’t) doing the measuring? — it also matters who is gathering & analyzing the the data, reporting out results & findings, and making decisions based on the data. Previous point about people’s differences in thoughts, feelings, behaviors, and patterns applies here. The key here is to involve the right “who” at each stage in data-driven decisioning to serve your goals. If you need to use your annual report to gain large-scale investment in your business or a big grant from a large institution, and you have a team of mostly (com)passion-driven, innovative, rule-breaking entrepreneurs — you might want to find a few advisors that have more guardian personalities and institutional leadership experience. If your aim is to make fast, agile decisions that support quick lean MVP production releases, don’t establish a diverse oversight group — give the team maximum autonomy and only involve only a few data advisors at key milestones (while ensuring customer feedback represents the diversity of your market).
Digging into the “Who?” often reveals and helps fix significant problems in the ‘what?’ ‘how?’ and ‘why?’ of a data and analytics system.
Including the right people in each part of a data system:
- Prevents failures based on misunderstanding your data’s relationship to the people you hope to positively impact
- Leads to insights & decisions more specifically honed to the people your work is most relevant to
What about you? What have you noticed about how data is or isn’t being used around you? What ways have you seen the ‘who’ impact the how/what/when of data and analytics?
Tune in for Part 2 of this series to learn more about what how what you’re not measuring might be key to making the most of metrics
