Does Fundamental Investing Work? Attempting to Predict Stock Success With Machine Learning

Like most of you, I have a strong interest in making more money and growing my savings faster. And as I’ve started my career over the last 2 years, I’ve been seeking advice on how to best manage the money I’ve earned.

To date, my personal finance strategy has been relatively conservative — invest the bulk of my savings in an S&P 500 ETF and let my them grow with the market. My friend Emmanuel who now works in asset management told me that this method — the Vanguard method — held the best blend of risk and reward, and he gave me a book called “The Boglehead’s Guide to Investing”, written by Vanguard’s founder John C. Bogle. This book explained that putting your money in a total stock market ETF was best because trying to beat the market is time-consuming and nearly impossible — even the best hedge funds could only beat the market for a year or two before ultimately providing similar, or worse, returns.

Yet still, there have been a few examples of people successfully beating the market time and time again to make extraordinary wealth. The most famous example is Warren Buffet, who has used a process of evaluating companies based on their fundamentals — financial metrics like net income and ratios like Earnings per Share — to make investments that have helped him become one of the wealthiest people on the planet. In fact, a whole school of investing exists based on picking stocks based on their financial fundamentals, aptly named “fundamental investing.”

Warren Buffet, the King of Investing, walking with a friend

Fundamental investing is based on the idea that you can find stocks undervalued by the market based on their fundamentals. You look at how the market rates a stock compared to how fundamentalists evaluate that stock according to a specific valuation metric. The logic goes that if you can find and buy undervalued stocks, then you can make money when stocks reach their market potential.

To test out whether fundamental investing principles really worked and see which specific fundamental metrics would factor most in predicting stock prices, I decided to build a model to try and evaluate fundamental investing.

Note: The next few sections concern the data analysis and modeling process. Feel free to skip to the takeaways below.

Data

I scraped financial data and valuation metrics from every single company in the Russell3000. The 2 time periods I chose to scrape from were the stock price, several financial metrics, and several valuation metrics from January 2018. I then scraped the stock price of January 2020 in order to calculate the lift over 2 years. Data was scraped from Morningstar and Yahoo Finance.

Thus, I had my independent variables — the financial and valuation metrics — and the dependent predictor variable — the percent a stock increased between Jan 2018 and Jan 2020.

Feature Engineering

After I collected the data, I engineered several ratios based on this article: “15 Financial Ratios that Every Investor Should Use” by investinganswers.com. Some of these ratios were:

1. Price to Earnings Ratio — The stock price / earnings per share. Gives an idea of how the stock is valued compared to how much that company is actually making.
2. Price to Book Ratio — Price per share / book value per share. Compares how the market values a stock versus how the actual shareholders value it.
3. Return on Assets — Net income / average total assets. Measures of how well the company uses its assets to make money.
4. Return on Equity — Net income / average stockholders equity. Measures how much money the company generates for each dollar given to shareholders. If a company generates more money for its shareholders relative to other companies, the stock will be more valuable to the market.

Modeling

After removing multicollinearity, I tried several different regression models to look at my data. Among these, I used regular linear regression, ridge, and lasso, using GridSearchCV to optimize parameters.

However, the model performances on these regression types were very low, producing an R² of .01 and less. (R² measures how well your features capture the variance in your target). This suggested that either the pattern was likely non-linear, or that the data simply had little relationship with the target variable, or both.

As a last ditch effort to get more out of my data, I created polynomial features out of all my existing ones and applied a Random Forest Regression model, which in a simplified sense uses “if, else” patterns to make predictions on the data. With this model I was able to improve the R² to .04 — a 400% increase although still quite poor.

Results

As stated above, the R² for these models was very poor, showing that the data had explained very little of the variance no matter what type of model and pattern I tried to use. In addition, I was able to reduce another performance indicator, RMSE, from 42% to 39%. But even so, the results still meant that on average, my predictions were 39% off. If a stock price went up by 50%, my model could predict that it only went up 11%.

In terms of silver linings, upon analyzing the feature importances of random forest model we see that all of the top 5 features contain 1 of 2 ratios: the Price to Book Ratio, and the Return on Assets.

Takeaways

1. It is very difficult to produce a machine learning model that can accurately and precisely predict stock performance from just fundamental ratios. Likely, many more factors contribute to stock price that are just as if not more important, which is why in my best model stock fundamental ratios only explained 4% of the variance in the target. Such contributing factors could be how well a stock has been performing as of late (technical analysis) or how well a stock’s industry has been moving as a whole.
2. This is not to say that fundamental analysis cannot lead to results — it just means that doing so might be enormously challenging. There’s a reason why most hedge funds don’t outperform the stock market each year.
3. In cases where fundamental investing does work, perhaps we’re looking for cases where outliers exist in key variables such as the 2 identified above — Price to Book ratio and ROA. With regards to Price to Book, if a stock has an abnormally low ratio, then the market values the stock much lower than its book value. Here, it seems possible that the stock price will increase so that the market aligns more reasonably with the book value. With regards to ROA, a stock with an abnormally high ratio may suggest that this particular company may be doing something special that allows them to get the most out of their resources, suggesting better future performance. However, this is just my guess, and I would need to do further analysis to see if there’s any relationship between outliers and stock performance.