In the machine learning world, generalization refers to the model’s ability to make accurate predictions from entirely new data points. And these generalized models can work on data from new scenarios. Human intelligence also works the same way. Children start learning from examples, initially, they usually fail to respond correctly to new situations; however, they gradually acquire generalization skills to respond to entirely new problems.
Generalization plays a vital role in intelligence building for humans as well as machines. People capable of quick abstraction are more intelligent and respond to challenging situations more smartly. In other words, the ability to generalize can be considered as the measure of intelligence.
An overfit model is just like a child who mugged up the results with no generalization ability. These models get a low loss during training but do a poor job of predicting with a new dataset. If a model fits the current data well, will it be able to make good predictions on the new dataset? Overfitting is caused by creating a model more complicated than necessary and eventually reducing the model’s ability to work with an entirely new dataset. The fundamental rule of machine learning fits the data in the most straightforward method.
If a cat’s picture is shown to the children, they can easily connect it with a cat. If the same picture is shown upside down, some of the children may have difficulty even recognizing it. And then the teacher needs to tell the child that this also represents a cat. Now the child realizes that shape of objects is independent of the orientation. When working with neural networks and deep learning, data augmentation is used to provide orientation independence.
Data augmentation
Data augmentation means generating new incarnations of the same data from the given data. Often this is programmatically done by modifying the images in the dataset with random flips and shifts. It makes the training dataset larger and helps the model generalize the shape of the object represented in the image and teaches the model that the shape is independent of the position and orientation of the objects defined in images. Precisely this is what teachers in kindergartens do to the child. So data augmentation helps the child and the model to generalize easily and learn fast.
Most of the Machine learning libraries provide an image augmentation API that can create modified versions of images in the training dataset just-in-time. Overfitting is the enemy of generalization since it makes the learner mug up without understanding. Data augmentation helps avoid overfitting by exposing all features of the object to the learner, whether it is a child or a deep learning model.
Model Overfitting
Overfitting occurs when a model tries to fit the training data so closely that it does not generalize well to new data. To summarize, if you cannot generalize well, your level of intelligence is low for humans and machine learning models.
Therefore, as you grow old, you master the technique of generalization, and your intelligence grows. The successful people sitting in leadership and strategic positions usually have this ability more than their team members.
Citation: Janardhanan PS (from datasciencecentral.com)
