The Role of Humans In ML Investing
One of my partners pointed me to a recent article that discusses the role that ML, specifically, deep learning, will play in investment management in the future.
The article spends some time going over what DL is and how it actually works, but the author also discusses how he thinks humans will work to prevent overfitting from the predictions that their DL algos spit out. He claims that the role of humans will be to set the investment management framework in which a deep learning algorithm will operate. Humans, essentially, will select the environment, deep learning will pick stocks within that environment, and because humans are setting restrictions on the universe of what can happen, overfitting will be prevented.
Now, I’m not so sure I agree with this statement, because that’s not really how overfitting works. But the author’s points led me to think deeper about something that I’ve been mulling over in the back of my head — specifically, how humans and ML algorithms will interact in the future.
I do think that humans will have a big role to play in ML-managed investments. Obviously, software will do the dirty work of crunching numbers, finding latent features, and discovering optimal decisions. But machines still can’t do everything, and I don’t think they’ll be able to for a long time.
Because of that, human input will be necessary to help them operate in proper context. Specifically, humans have common sense and the ability to judge something on subjective basis. The former is very useful, and not something we can teach a machine quite yet. The latter can be useful, though it can also lead to a host of problems, including confirmation bias and seeing patterns where none exist.
However, because humans do possess these attributes, they can begin to inform the development and operation of ML algorithms that don’t take these things into account.
Human analysts can begin to work with data scientists to discover new or better features that need to be included in an algorithm, features that are specific to how humans analyze stocks. Humans can also check the output of algorithms to make sure that they aren’t finding obvious or simplistic patterns. Similarly, machines can be used to verify whether human predictions are objectively looking at the right factors, or whether human bias is coming into play.
In the article I mentioned above, the author also mentioned that humans will be responsible for finding and cleaning the data that will be used by algorithms. I believe this will be one of the most important things that humans will do in the future.
The use of various datasets is still a subjective matter. Humans have ideas about what sorts of data will be useful, and it’s up to the humans to provide this data to the algorithms to see if it proves predictive. This is an area where human subjectivity can play a crucial role, and the right researcher or analyst can make a world of difference.
In the far off future, the one that Elon Musk is concerned about, humans may end up being completely eliminated from this space. But in our lifetimes, I believe, humans and machines will work together to come up with actively-managed investment strategies that neither could beat on their own.