A First (And Close) Look at Our Portfolio Data Analysis Report
Foothill Ventures is a $150 million fund that primarily invests in seed-stage companies with a preference for technical founders and cross-border operational experience.
Every six months, we conduct a fairly thorough strategic review and portfolio review. This summer’s review happened to coincide with the arrival of our summer intern class, and we took advantage of the additional manpower to do a far more thorough review of our performance data and decision-making process than we had conducted in the past. We present the summary below for people interested in a quick-take, but the whole (27 page) report is available here for people who really want to geek out :).
The good news is that our funds are performing extremely well: our returns are well into the top decile of US funds.
When reflecting on these great results based on Pitchbook’s TVPI benchmarks, the obvious questions we had were: what led to our winners? Could we have predicted them from the time that they were sourced? And, what led to our laggards? Could we have avoided these investments?
Fortunately, we started collecting data that would help answer these questions several years ago: for every new company that we see, we fill out a “fit scorecard” (that acts as an initial screening tool) and a “valuation scorecard” (that acts as a predictor of future value). The fit score ranks how closely a company matches our investment thesis and preferred company profile, while the valuation score ranks the company and founding team based on a rubric of important qualities, such as strength of the management team or technical expertise. For anyone curious, we also published a more detailed breakdown of the exact rubrics that we currently use.
Our task was to analyze the individual components of these scorecards to see if any of them have predictive power for exit multiples.
As with all exercises involving prediction, this project faced a few assumptions and potential errors. Among the most obvious is that most of these companies haven’t yet exited, so TVPI mostly exists on paper. To partially confront this issue, we have a tradition of voting on the predicted exit multiples of each holding in the portfolio at each semi-annual session. At this point, we have several years of predicted values for each company, in addition to its most recent “marked to market” value.
Both of these numbers are meaningful: some companies haven’t been marked up (because they haven’t yet raised another round of capital), but — as insiders — we know that they are progressing extremely well, and should have a higher exit multiple. On the flip side, some companies have not been officially marked down, but we know that they are struggling, and so are very bearish in our prediction.
When approaching our valuation scorecard and voting multiple data, we used three main avenues of analyses. First, we compared differences in valuation scores between our top 10 and bottom 10 performing portfolio holdings to identify key differences and trends between our winners and laggards. Second, we conducted linear regressions on our valuation score and voting multiple data for various sectors of our portfolio, and evaluated which categories of the rubric were most predictive of voting multiples. Lastly, we created a logistic regression model to both identify key predictive qualities in our portfolio holdings while also creating a predictive tool that could potentially be applied to future companies we look at.
Here, we include a brief look at a snippet of our analysis for the top and bottom 10 performing portfolio holdings (if you want an in-depth understanding of our scorecard, please click here):
Approach 1: Visual Analysis of Trends
This graph on the right shows the top 10 (red) and bottom 10 (blue) portfolio holdings in terms of the voting multiple. As shown, there is a clear visual difference between their valuation scores. We then break down the difference by category.
This visual indicates the ordered differences in the valuation rubric between the top and bottom 10 portfolio holdings. We expect Total and Management scores to have the largest differences as they are a sum of many individual components. For the individual components, we find that the top three are all management-related; five out of nine management scores are significant; and two out of six market and product related variables are significant.
A clear takeaway is that the management team has an overwhelming contribution to the differences for our top 10 performing portfolio holdings, which we can apply to reweigh our scorecard in the future.
We dive more into the analysis, implications, and limitations in the full version of our article.
After completing our three stages of analyses, we found several significant conclusions that will help us improve our company’s investment procedures. From our valuation scorecard rubric, we find that past indication of a founder’s success is the single most important predictor of an investment’s voting multiple, with other qualities such as the founding team’s management strength and technical expertise being key predictors. We also find that several categories in our valuation scorecard that we would expect to be important predictors are not as statistically important for our portfolio holdings’ performance as we expected; these conclusions allow us to adjust our scorecard weights accordingly. Lastly, we discovered that we constructed our valuation scorecard to fit well for qualities important for software companies but less so for hardware and biotech companies, suggesting the need for rubric revisions and some process improvements.
This summary exists to lay out some of our analyses and a brief list of our main conclusions. If you would like access to the full report along with our data visualization (all of the analyses over 27 pages), click here. We also attached a brief sample of our analyses from our linear regression section below.
Approach 2: Linear Regression
After conducting visual analysis on the top 10 and bottom 10 performing companies, we then attempted to identify if a statistically significant linear relationship between valuation scores and voting multiples existed across the entire portfolio. However, before conducting this analysis we needed to ensure that all rubric categories were equally weighted before the analysis, in order to identify which categories were most important.
Our current rubric assigned different point values to different categories, based on our perceptions of their relative importance. For example, a founder’s history of related experience was ranked on a scale of -5, -1, 0, 1, 4 from lowest to highest, while the product’s technical uniqueness was ranked on a scale of -15, -10, 0, 10, 15. Because of this, certain categories are already weighted higher than others, which could potentially mislead any analysis about relative category importance. The following table shows all current rubric categories:
Figure 2a: Table of Scorecard Rubric Categories
To unweight the categories, we assigned every category to have a 0 through 4 scale, with 0 as the lowest score and 4 as the highest. With all categories now with the same distribution, we then recalculated the unweighted valuation score for each portfolio holding and conducted a linear regression of voting multiple to the unweighted valuation score, composed of each individual category.
Figure 2b: Regression Output of Voting Multiple to Unweighted Valuation Score for Entire Portfolio Data
We can draw some key conclusions from this regression output. First, the R-squared value is 0.4462, which suggests that a weak linear relationship may exist between unweighted valuation score and voting multiple. A R-squared value of 0.4462 is not statistically significant from a theoretical perspective, but a value of 0.4462 obtained from real-world data, along with a clear positive relationship between the two variables in visual graphs, suggest that there is some predictive value in the unweighted valuation scorecard.
Second, most of the factors have high p-values that prevent us from concluding that there is a clear linear relationship between specific categories and the voting multiple data in our sample. Most p-values are above 0.10, with only previous founder success having a low p-value. Lastly, several confounding factors with unintuitive results exist in this regression using all the scorecard categories. For example, the coefficients suggest that less founder risk, less team functional experience, and smaller sizes of business opportunity are correlated with higher voting multiples, which seem like illogical relationships. To understand these two issues in more detail, we performed a further dominance analysis on the rubric categories, which ranks each independent variable by their relative contribution to the overall R-squared value and regression.
Figure 2c: Dominance Analysis of Voting Multiple to Unweighted Valuation Score Categories
When evaluating the dominance analysis results, we can observe that the confounding factors we previously identified such as team functional experience, founder risk, and size of the business opportunity are some of the least impactful and correlated factors. Some of the most important categories include founders’ history of past success, the competitive landscape of the business, and components of the overall “management score,” such as technical experience, executive experience, and completeness of the management team. These match our initial hypothesis of the most important factors, since we can observe a general linear relationship between the voting multiple and management score.
There are some surprising results as well. Notably, the ability to operate cross-border, which is an important part of our overall investment thesis, does not seem to be highly correlated with voting multiples at all. This conclusion is in stark contrast to our analysis of the top and bottom ten performing companies, where the ability to operate cross border is within the third most significant factor. We hypothesize that the inability to operate cross-border may be an indicator of low returns, but not a distinguishing factor between medium and high returns. This result may also be due to the fact that most of our portfolio has some ability to operate cross-border so there is not a meaningful sample size for both groups, rendering its predictive power to be low.
Some conclusions:
Given the “power law” skew in venture (where a very small number of holdings produce the vast majority of returns), the relationship between our initial valuation prediction and our “top 10” actual holdings is worth digging into.
Our initial weightings seem off: we should be weighting “Management Strength” more heavily, and exploring this bucket more thoroughly. For the time-being, we will continue to just collect data and analyze results, rather than using the input as a key part of our decision-making.
We look at this exercise as being about a process. We will continue to collect observations, make hypotheses, analyze results, and adjust. We believe that it is important to be very transparent about the usefulness and limitations of the approach, and will publish our results regularly. We welcome input from people interested in the subject of early stage venture scorecard methodologies, and valuation predictions.
+++++++++++++++++++++++++++++++++++
As noted above: to access the full report, click here to get to the download page
