Unsupervised Learning — The New Hottest Thing on the Block

Unsupervised learning is a new hottest thing on the block, and it definitely beats traditional statistical analysis to a pulp. In this article, we’ll take a look at what unsupervised learning is, how it works, and why it’s so effective.

What is Unsupervised Learning?
In machine learning, there are two main types of algorithms: supervised and unsupervised. Supervised learning algorithms learn from labeled training data, meaning that for each example in the training data, there is a corresponding label (or target variable) that the algorithm is trying to predict. On the other hand, unsupervised learning algorithms learn from unlabeled training data. This means that there is no specific target variable that the algorithm is trying to predict. Some researchers, myself included, argue that the self-extracting knowledge of unsupervised learning is exactly what comprises artificial intelligence, or AI.

So why would we want to use an unsupervised learning algorithm? Well, there are two main reasons. First, unlabeled data is much more common than labeled data. Second, even if we have labeled data, we might not necessarily know what we’re trying to predict. For example, suppose we have a dataset of images of handwritten digits (0–9). We could use a supervised learning algorithm to train a model to classify those images into their respective classes (i.e., 0–9), but what if we don’t care about classification? What if we just want to learn something about the structure of the data? That’s where unsupervised learning comes in.

How Does Unsupervised Learning Work?
There are many different types of unsupervised learning algorithms, but they all have one thing in common: they try to find some structure in the data. For example, suppose we have a dataset of people’s heights and weights. An unsupervised learning algorithm might cluster those people into groups based on their similarity in height and weight. Or suppose we have a dataset of images of faces. An unsupervised learning algorithm might try to find groups of similar-looking faces.

Why is Unsupervised Learning So Effective?
Unsupervised learning algorithms are able to find hidden patterns in data that would be impossible to find using traditional statistical methods. For example, suppose we have a dataset of stock prices over time. We might use an unsupervised learning algorithm to find hidden patterns in those stock prices, such as whether there is seasonality (e.g., stocks tend to go down in December) or whether certain stocks tend to move together (e.g., stocks in the same sector tend to move up or down together).

Unsupervised learning is an exciting new field with huge potential. If you’re not using unsupervised learning algorithms in your business or investment decision-making process, you’re at a serious disadvantage relative to your competitors who are using them!

Irene Aldridge is President and Head of Research at AbleMarkets and AbleBlox, Editorial Board Member at Journal of Financial Data Science and an Adjunct Professor at Cornell Financial Engineering, where she teaches Unsupervised Learning, among other courses. She is also a co-author of Big Data Science in Finance (with Marco Avellaneda, Wiley, 2021), a book that covers unsupervised learning in great mathematical detail.