The imperative of intuition + data for better business analysis
In a new article for Modern Analyst, titled The rise of the data-inspired BA, I illustrate how high-performance business analysts are leveraging data — big and small — to take the emotion and opinion out of decision-making and find the opportunities with the greatest potential for growth.
I adopted the term “data-inspired” instead of “data-driven” for a reason.
As a business analyst turned data scientist, I’m well aware that data can create invaluable competitive advantage, helping organizations get better at things from predicting problems with demand and supply chains to achieving high rates of perfect orders and understanding what triggers will make people buy more. In my previous job, I worked on data science projects with obvious goals, such as improving the position estimates of assets tracked via cheap GPS sensors so that construction sites and ranch owners could better track the location of expensive equipment and animals at a lower cost.
Still, high performing BAs also accept that the analysis-versus-instinct debate is useless, and that we need both to succeed. Indeed, none of the best business analysts I know are fact-or-feeling purists. Rather, they combine analytics with intuition to decide when data from the past can be used to innovate, or when a new idea that comes purely from instinct should be put to the test in order to generate the evidence needed to convince decision-makers of its value.
How to become an analytics-minded business analyst
As a growing number of companies embrace analytics as a strategy and start using data to optimize supply chain flows, reduce inventory and stock-outs, identify customers with the greatest profit potential, etc., BAs will have to adapt to this new business reality.
A company-wide adoption of analytics requires changes in culture, process, behavior, and skills. Thriving in this kind of environment requires, in addition to business and relationship skills, both an appreciation of, and a familiarity with, quantitative methods. While you don’t need to become a statistician, you must have capabilities in the following areas:
- Understanding of quantitative techniques so you can recognize their applications and limitations;
- Awareness of peculiarities and shortcomings of data such as missing values, duplication, inconsistency, and other quality issues;
- Ability to explain analytics findings in plain language.
Effective quantitative decisions are not about the data; they are a result of deep, trusting ties between the producers and consumers of analytics. Since many business analysts already have robust “social skills”, a willingness to pick up the skills listed above, and to team up with data scientists and quantitative analysts, can create big opportunities for career growth.
Why data can’t replace intuition
As anyone who has accumulated experience as a manager or knowledge worker knows, while analytics can be of great value, relying exclusively on data for decision making is not always practical or wise. Some times we do have to resort to intuition for the best results.
A classic example is Apple’s investment in creating retail stores, which I mention in the article Scientific analysis for innovation: not an oxymoron. The initiative required making a leap of faith that a branded retail experience would create rather than destroy value. Data came later, to confirm the opportunity was there.
Gary Klein, PhD known for his cognitive and insight models, writes in his book The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work:
We shouldn’t simply follow our intuitions, as they can be unreliable and need to be monitored. Yet we shouldn’t suppress our intuitions either, because they are essential to our decision making and can’t be replaced by analyses or procedures.
The right data supports better choices and lower risks, but true innovation that leads to above-market growth often requires a blend of analysis and intuition. Examples of when it’s not a good idea to ground decisions entirely in analytics include hiring and marketing decisions. While those kinds of decisions are supported by analytical tools, statistical analysis can only take us so far. Accurate assessment of personality, character, and culture fit, or whether an ad will incite emotion and stand out, still require subjective assessments.
Data + intuition lead to the best business analysis results
A well-rounded BA can use their intuition and creativity to help their organizations identify business problems to solve or opportunities to exploit with the available data. For example, a business analyst from a bank had an idea to combine information from ATM transactions, online queries, customer complaints, and other disparate sources to identify duplicative interactions that were eliminated to reduce costs and improve the customer experience.
While fact-based decisions should be universally encouraged in organizations, to bring innovation to the market, we often need to start from an intuition about what is going on in the business, where the risks, issues, and opportunities may be, which products to develop, or which future direction to take. That doesn’t mean we should just run with that intuition — we need to transform it into a hypothesis to be tested via experiments, gathering and analyzing the data. But it’s that combination of intuition and hard data, of creativity and machine capability, that we keep seeing give rise to the greatest success.
