This Is How Machines Learn! Supervised Learning (Part 2)
Our goal with this series is to enable everyone to understand AI phenomena in their daily lives, as well as to actively shape the growing influence of AI on our society. Therefore, we do not consider any technical details or provide an introduction on how to use certain machine learning frameworks. Instead, we focus on explaining the underlying ideas of machine learning which empower everyone to understand and shape the digital world that surrounds us.
For many children, a dog is the animal they ever encounter (“a bow-wow”). Initially, a child will apply this term to other animals as well, such as cats and cows. Through further encounters, the child will then learn that these other animal species have distinct names and it will soon be able to identify the animals by their individual characteristics, even without explicit description.
Underlying Idea
Supervised learning algorithms function similarly. A series of data with corresponding labels is given as input. The goal is to find a pattern by which the correct labels can be assigned to the data. Subsequently, this pattern, stored in a model, can be applied to new data.
The steps below describe the activities in the figure.
① The availability of labeled data, also called training data, is crucial for the use of supervised learning. This data can take the form of bricks labeled “A”, “B”, “C” or “D”, like in the example, or photos labeled “cat” or “dog”. Sometimes existing data sets can be used. If this is not the case, data needs to be labeled manually.
② Using this labeled data, the algorithm establishes its own pattern that uses the features of the data (e.g. the shape and color of bricks) to assign a label (e.g. “A”). For images, for example, clever techniques can be used to identify simple geometric shapes that serve as features. Since the labels of the training data are known, the learning process can be “supervised”: The procedure receives feedback as to what extent the pattern labeled the data correctly. Based on this feedback, the pattern is gradually refined to achieve better and better results. This step is also called training phase. The model (the wooden stencil, in the case of our robot) represents the pattern, which should ensure the input data is given the correct label. This pattern could be represented explicitly as a decision tree, or implicitly as the parameters of a neural network. In practice, satisfactory results usually require a large amount of training data, e.g. several thousand images of animals, each labeled “cat” or “dog”.
③ Once the training is complete, the model can be used to label new (similar) data. The robot can now, for example, use its wooden stencil to assign the label A or B to other bricks. A supervised learning model trained to distinguish cats from dogs in photos can now also be used to label unknown images of dogs or cats, even if the viewing angle or lighting conditions differ from those in the training set. However, before such a model can be used, its performance, i.e. its accuracy, should be determined. For this purpose, retaining some of the labeled data as “test data” would be a good idea. This allows us to test how well the algorithm labels data that has not been involved in training. The accuracy required will vary according to the intended application. In order to predict whether a customer will click on personalized advertising, 60% correctly labeled test data might be sufficient, while for the recognition of images, an accuracy of 90% and more would be desirable.
Even if the algorithm can generalize from the specific examples provided, it is still not prepared for all eventualities. Our robot, for example, would give a semicircular brick the same label as a circular one. How could the robot be expected to know that a semicircle should have a different label if it has never seen a semicircle before?
Application Areas
A large number of commercially-used AI applications are based on supervised learning. Common application areas of this learning paradigm include classification and regression problems.
In the case of classification problems, the algorithm learns, like in our robot and image recognition example, to sort data into different (predefined) categories, i.e. to assign a label to them. Typical areas of application include: Does a photo show a cat, a dog or a bird? What risk category does a debtor fall into? Is an e-mail to be classified as spam or not?
Supervised learning is also used when data should be assigned a numerical value rather than a predefined label. In so-called regression problems, the algorithm determines the relationship between data points and labels provided as numerical values. Regression is therefore used to address questions like: How many weeks until a user cancels his or her video subscription? What price can we sell the house for? What will be the share price gain?
This concludes part 2 of our series. Thank you for reading. If you have any questions, feel free to ask them in the comments.
Go to part 3.
Written by Stefan Seegerer, Tilman Michaeli and Ralf Romeike.
The robot is adapted from https://openclipart.org/detail/191072/blue-robot and licensed under CC0. The article and the derived graphics are licensed under CC-BY-SA.
