If data is the answer, what’s the question?
Data doesn’t make decisions
“Write the next chapter” has always struck me as one of Reece’s most embedded values, second only to “Creating customers for life”. It permeates every corner of our organisation; from the branches’ direct involvement in co-designing software, to our investments in new services — working on the next chapter at Reece is an assumed way of life.
Building the strategic agenda for data products at Reece, however, requires we evolve our instincts. Knitted in industry observations of barriers to digital transformation are the challenges organizations must surmount when it comes to data & AI investment — with frequent calls for addressing culture and capability at the root of change.
Experience highlights that data accessibility is a confounding factor: the ease with which anyone with a laptop can become a data practitioner — may at the same time mask what it takes for organisational mastery. So teams often assume well-trodden product frameworks work the same for delivering data products, only to be surprised at outcomes that fall short of expectations.
It has, suffice to say, been a year of trying, learning from mistakes, and finding ways to turn instinct into substance, then substance into execution. Here are some perspectives I’ve gathered along the way.
Good data and smart machines aren’t enough
Data-enabled strategy is about making decisions smarter and faster, in a way that generates business advantage. As technologists, our instincts quickly gravitate to the ‘smart’ and ‘fast’: data capturing the depth and richness of our business, and near-limitless possibilities with cloud and ML infrastructure.
But there is a third, perhaps less-emphasised element, related to the decision-making itself. Decisions, to suggest a simple definition, are the output of reasoning through input information. While the need for quality data and sound technology remains, it is an organisation’s capacity to reason that decides its strategic success.
To unlock value from data, we must understand our (own) decisions
An intrinsic part of modern product execution is user feedback, whereby iterative development relies on feedback such that each iteration is improved on the last. In this, a core assumption is that the feedback received makes the next iteration better than the last.
Curiously, with data products this assumption isn’t as safe as it is with less data-heavy software applications. Be it that ‘discovery’ phases of machine learning feature development never seems to finish, or that dashboards don’t get used as pretotyped — something about delivering data products just seems more fiddly.
It helps to study what the feedback is collected on. User feedback is reliable when the underlying reasoning is well formed. This tends to be the case when feedback pertains to tasks, because users who know their jobs will know the tasks needed to get the job done. For instance, branch managers know well the tasks related to generating quotes — these tasks don’t materially change after feedback has been shared with product teams.
In contrast, data products tend to command a much larger options space. Whereas a web application’s jobs-to-be-done might decompose neatly into testable features, the same may not be analogous for data products — even routine single-metric decisions call for systems-thinking.
Take for instance, decision-support tooling to uplift Gross Profit (GP). In this scenario, effective feedback requires users to know exactly how to lift GP every time, using the swathe of operational levers available to them in order for their feedback to be reliable. Further, these levers are not easily separable — it’s not straightforward to test a product mix hypothesis without accounting for price/volume trade-offs. And so, data product development often demands more of users’ reasoning capacity in order for them to be reliable witnesses to their own needs.
If data is the answer, what is the question?
Grasping the criticality of reasoning is not only useful for understanding our past missteps, it’s surfaced important considerations for executing on data-enabled strategies going forward.
An organisation’s capacity to reason is a prerequisite for realising value from data products, and must be prioritised. Save for a few notable exceptions, machines don’t reason better than humans teach them, and dashboards certainly don’t make decisions. Building into our roadmaps the effort to help users’ grasp of their own reasoning process has been a game changer for product adoption. We call this an “enable-first” approach.
Because an enable-first approach is user-enablement focused, delivering data products becomes more akin to a dialogue. It depends heavily on a user’s clarity around how they make decisions –– a clarity that grows as the product development process uncovers their reasoning. Early in the development lifecycle, user feedback must often be refined, and in some cases (respectfully) parked. Because folks aren’t familiar with what data we have, or what data can and can’t tell them, their initial feedback might not be actionable. “This model is great but isn’t telling me what our customers aren’t buying from us”; and quite frankly, it never will. What is needed, if data is the answer, is first a systematic approach to understanding the question.
Concluding remarks
All of this supposes that users are willing partners on the enablement journey to begin with. To this end, working with teams who live and breathe “Write the next chapter” makes building data products at Reece an absolute joy.
Taking stock of the year that’s been, 2021 for our practice has been an incredible year of wrapping our heads around where Reece’s most consequential decisions lie and putting down roots. After a well-earned rest, the year ahead looks even more exciting as all signs point now to translating our learnings into better, faster and more robust decisions. We are always on the lookout for talented folks who might join the mission. If that’s you, drop me a message.
