Startups and Sabermetrics
The fallacy of comparing startup investing to “Moneyball”
Over the past few years, a relatively large number of ideas, strategies and concepts have drawn comparisons to “Moneyball,” and in the startup ecosystem, there has been no exception. The volume of comparisons went on steriods earlier this year when famous entreprenuer Steve Blank told us, “It’s Time to Play Moneyball”
But, as someone who has spent time in both professional baseball and at a startup, I find that likening an [often great] idea to Moneyball is typically good marketing around an apples-to-oranges comparison.
The main differences between Moneyball and Steve Blank’s ideas are three-fold:
- Baseball statistics provide a far superior dataset to the startup/VC/accelerator dataset:
The volume of baseball data is simply enormous. Fairly reliable statistics exist for nearly the entire 20th century, and Bill James’ Baseball Abstracts gave birth to an entire baseball-related sub-field of statistical analysis called Saber-metrics nearly 40 years ago. Famously, Billy Beane executed on the analysis and insights made by Sabermetricians like James, who paved the way for data-driven innovation in baseball management (Bill James and Paul Depodesta, who brought James’ thinking to the A’s, deserve as much or more credit as Beane).
Specifically, the A’s were (somewhat remarkably) the first to realize that building a roster is an optimization problem. By focusing on maximizing for outputs like runs scored per dollar of salary, they found that metrics like OBP (on base percentage) and OPS (on base + slugging), were better indicators of team success than both traditional quantitative metrics like Batting Average, and traditional qualitative measures like speed, arm strength, confidence (as inferred by attractiveness of players’ girlfriends), etc. [^1]
When it comes to optimizing a baseball roster that consists of major league players in their prime, the sheer volume of data makes the math difficult to refute.
But for startups and venture capital, data transparency has harldy been the norm. When Bill James began publishing his wildly popular Baseball Abstracts, venture captial was still so opaque that simply being able to find that a VC firm existed was a sign that an entreprenuer might indeed be competent.
Further, it’s often not clear which factors wind up leading to product-market fit and eventual breakout success, even with full data transparency. Steve Blank, Eric Reis, and the Lean methodology have increased the scientific nature of startup analysis, but comparing that relatively small data set to baseball’s massive and time-tested data bank is simply incorrect.
2. The use cases for data could not be more different:
In baseball, a “Moneyball” approach dictates that a team looks primarily for statistical consistency and predictability and optimizes their roster for whichever statistics are deemed to increase the odds of expected outcomes. In the case of the A’s, management optimizes for a lineup of hitters that will score the most runs for the least number of dollars over a 162 game season.
Conversely, the biggest “wins” in startups come from outliers, which by their nature are inherently difficult to predict. Everyone in the startup community is well aware that Y Combinator, the accelerator that uses the most data of any accelerator in [recent] history, still has 3/4 of its portfolio value comprised by just 2 companies (DropBox and AirBnB).
“Black Swan Farming” and optimizing a baseball roster are simply two different endeavors.
3. “The best thing about baseball is its tradition, and the worst thing about baseball is its tradition:”
The biggest flaw with the Baseball Industry->Startup ecosystem comparison relates to what most folks missed as the biggest story related to Moneyball: In the modern era of baseball, Moneyball has been literally the ONE innovative and risky management strategy to evaluate talent and compile a roster.
While there are certainly symptoms (perhaps even large, systemic symptoms) of groupthink in the Silicon Valley investing ecosystem, the extent to which it exists among investors and accelerators pales in comparison to the groupthink in baseball management. The phenomenon is so pervasive in baseball that it could only possibly exist on such a large scale in an industry that is explicitly exempt from US anti-trust laws, and subsequently, not governed by typical market forces.
Furthermore, the status-quo prior to Moneyball created an interesting paradox: while the mere creation of accelerators like Y-Combinator was a more innovative idea than Moneyball in its entirety, the risk Billy Beane took implementing Moneyball was incomparably greater than the risk an accelerator or VC would assume in adopting Steve Blank’s recent advice.
The idea that accelerators should use data is hardly revolutionary. Great startups and companies make data-driven decisions; the best investors use data to build and execute a thesis, and it’s logical progression that the best accelerators should do the same.
The real “Moneyball” question is therefore not whether accelerators should use data (a valid argument) but what specific data points, if any, would actually predict improved outcomes. Comparing Blank’s ideas to Moneyball implies that he’s already found the answer, something that will take several more years of experiments, results and analysis to determine.
Notes:
^1 Saber-metric research has since shown that its own predictive power is actually quite limited until a player reaches his prime around age 26, and qualitative measures are still a necessity in the minor leagues to evaluate young talent.
In this respect, the challenges faced by MLB teams during the first-year player draft and those faced by accelerators are indeed analogous. But, as stats don’t hold nearly the predictive power for college and high school prospects as they do for established big leaguers, the “science” of the MLB draft is hardly an enviable goal for accelerators.
