On paper, the numbers are great, but there are factors that can’t be quantified. The case for qualitative analysis in a quantitative world.
Let’s go back: it’s summer 2015. A simpler time. That “Want to Want Me” song won’t stop playing, over and over again, from every radio on Earth. The phrase “fake news” has yet to enter common parlance.
And in every publication is a profile of a young woman, often accompanied by a photo of her in a black turtleneck, studying a tiny vial holding an even tinier drop of blood. You know, the picture from the featured image.
This is Elizabeth Holmes.
As CEO of medical tech startup Theranos, she’s doing pretty well (remember, summer 2015). The company claims that it can run a full range of lab tests on a tiny fingerprick’s worth of blood, and it’s raised over $750 million, with a valuation of over $9 billion.
Needle-shy investors have no reason to doubt the company, which has just signed a pivotal deal with chemist chain Walgreens and received FDA approval on one of its tests.
Its board includes everyone from past secretaries of state to generals to a former CDC director. On paper, this is a company on an upward trajectory towards wild, game-changing success and innovation.
Sure, these investors shied away from some of the tougher questions, but this is an established company run by a young visionary. And if the numbers check out, which they do, what do they have to lose?
Quantitative vs. Qualitative
The due diligence process, like all company and market analysis, is based on finding patterns, connections, and discrepancies in data. The most effective analysis requires a thorough examination of two types of data: quantitative and qualitative.
Qualitative data is information pertaining to descriptions, characteristics, qualities, feelings, and anything else that can’t be counted. It answers the “what” and the “why” (what makes people buy a competitor’s product and not ours? Why are our sales figures so low?).
Collecting this information usually means taking a deep dive into the subject matter. Researchers can find it by asking open-ended questions, going through content, conducting interviews, and making observations.
Quantitative data is anything that can be put into numbers. It’s the answer to “how many,” and can be used to find connections between variables (for example, the link between company growth and revenue).
You can get this information through closed-ended questions, surveys, and, increasingly, using AI-based tools that sift through the internet and databases for numerical data.
The Quantitative in Action
When we’re analyzing startups, there are questions that our analysts have to ask. For instance:
- How big is this company’s market?
- How saturated is it with competitors?
- How closely do these competitors resemble the particular company’s niche?
The answers are calculated through an algorithm and are charted on a graph. Here’s the Zirra competitor breakdown for Instacart.
That’s some quantified analysis right there.
It’s also crucial, of course, to check employee growth over time, and in which particular sectors the company’s expanding.
LinkedIn knows what we’re looking for, so it auto-generates a handy chart of its own.
For most budding startups, today’s numbers can project tomorrow’s revenues:
- Company age
- Employee growth (in which sectors?)
- Funding rounds (when? How much? By whom?)
- Market size
- Market trends
- Profitability
- Web traffic
- Number of competitors in a space
A bit of qualitative analysis will always be necessary to make sense of the full picture — if a tech startup is beefing up its marketing, PR, and HR teams, trouble might be brewing — but a quantitative overview of legal data might tell the same tale.
With the future of investment research dashing towards automation, the more information that can be extracted, quantified, and put through an algorithm, the better.
Skimping on Qualitative Analysis
If the world were perfect, if every company or investor had a research team of 30, if every database and source were open to the public, and if money were no object, quantitative and qualitative analysis would live in perfect harmony. They would complement each other to form a balanced and thorough evaluation.
But that ain’t life.
The company analysis process often begins and ends with quantitative analysis — and that makes sense.
In these modern times, there isn’t much that can’t be synthesized into numerical values and algorithms. Starred reviews turn sentiment into statistics, and algorithms can be mobilized to recognize our faces in a crowd, scout out fake online profiles, and find us dates that even our moms will approve.
So if quantitative data can do all that, of course it can give us a pretty accurate picture of the health, risks, and viability of a company or market.
That’s how we end up in a situation where analysts and insights services tend to skimp a bit on the qualitative.
Most business intelligence tools on the market, like Pitchbook and CB Insights, are really just databases selling access to quantitative data for a pretty penny.
Clients, their revenues show, are willing to pay just to get their hands on a company’s numbers, and leave their research process at that.
Sure, prioritizing quantitative data makes research easier to delegate to the BI tools and/or robots in our lives, but there’s another reason we’re so keen on quantitative data: math is the only thing we trust.
We hear numbers and statistics and we think objectivity, accuracy, and reliability.
There’s no such thing as a subjective number, and a formula can’t bully us with sales tactics.
The Downside to Quantitative Analysis
You’ll hear that numbers don’t lie; people do.
While that’s generally true, there’s a pretty glaring issue with that statement. The numbers always come from somewhere.
Since we haven’t reached the singularity yet, they usually come from people. And, as we established, people lie.
There are a lot of ways that lies can manifest into numbers for companies.
Sometimes it’s maximizing sales by skirting compliance. Sometimes it’s selling “vaporware” — collecting round after round of funding by touting a product that doesn’t, and may never, exist. Sometimes it’s bad bookkeeping, or little white lies to the board or investors that pile up.
And sometimes, in the case of Theranos, it’s a little bit of everything.
Their numbers were excellent — according to some valuations, it was a rare decacorn (a startup worth over $10 billion) of the unicorn age. But those numbers were acquired through layers and layers of first-class deception.
The Human Element
Elizabeth Holmes sold Theranos to investor after investor, by boasting:
- its technology’s place on the battlefields of Afghanistan (false)
- its ability to run multitudes of tests on one patented machine (false)
- its prototype’s effectiveness at testing just one drop of blood (also false)
And the cash pumped through its veins.
Few of these investors, though, were interested in the big-picture qualitative data. How the patients who’ve used Theranos’s services feel about it, for one, or the expert opinions of scientists in the field.
Investors handwaved medical and scientific concerns and calls for peer review. They ignored first-hand accounts of wildly inaccurate test results.
As tech industry veteran Jean-Louis Gassée pointed out in his own tale of Theranos woe, none of the investors, journalists, or board members caught in the Theranos web actually bothered to get a Theranos blood test themselves.
John Carreyrou’s pivotal investigation broke in October 2015, and Theranos began collapsing in earnest in 2016. By then, investors and tech journalists were asking what went wrong, how so many people could have been taken in by Theranos and Holmes’s fraud.
The answer was clear: they trusted the numbers, they didn’t ask the right questions, and they didn’t take the qualitative data into account.
The (Qualitative) Questions to Ask
Impressive as this technology supposedly was, did experts and professionals give it their seal of approval? The answer was a resounding “no.” Holmes may have made grand promises, but the science doesn’t check out.
Here are a few questions that could have saved Theranos’s investors a $9 billion headache:
- What is the founder’s background?
- And how does their expertise pertain to the field? Holmes was a promising college dropout with no biotechnological experience.
- Who’s on the board of directors?
- Even if they’re big names, do they have knowledge of this particular field?
- What do customers and industry experts have to say?
- How about Glassdoor reviews, to hear from current and ex-employees?
- Do non-competitors in the same industry believe that the concept is viable, or even possible?
The numbers should confirm what all of these sources are telling you and vice versa. If they don’t, the information you’re collecting might just be pointing you to some hidden truths.
When we at Zirra put together our Screening, Analysis, Due Diligence, and Market Analysis reports, we make sure that the story checks out.
We take a deep dive into the founders’ backgrounds, and the board members. We listen to what customers, employees, and industry experts have to say.
Using AI to collect the numbers and analysts to give them a human touch, Zirra is able to make sense of company fingerprints in a way that more expensive databases can’t.
Learn more about what we do from our website, search for any company and get an AI-generated report in minutes, for free, or order a full analysis report.
Let this be a cautionary tale. Sure, human opinion is fallible, but expert opinions can be exponentially more useful than revenues and returns.
Polls can be inaccurate and off-base, but badly-sampled statistics can cause even more damage. Tempting as it is to let the numbers speak for themselves, we need to remember that behind every number is an imperfect person. In your company and market research, don’t fear the human element — it might save you millions.
Originally published at wisdom.zirra.com on August 9, 2018.
