Overfitting (What They Are & Train, Validation, Test & Regularization)
I’ve taken a bit of a hiatus because I’ve been busy with school and other things and I’ve been spending time learning more ML/DL stuff. Here’s another article! We’ll explore what overfitting is, why it is bad, and the solutions that exist to reduce it.
What Is “Overfitting”?
We must first review what a typical machine learning task looks like. I explained in a previous article that, in a typical machine learning task, the researcher divides the dataset. There are two methods people like to go about doing this.
Train & Test
This method is simple: divide the data to a separate training set and a testing set. The training set is where the machine learning algorithm learns from and the testing set is the one used to evaluate the performance of the program. I like to keep a 4:1 ratio, 4/5 of the data dedicated to training the program and 1/5 of the data dedicated to testing the program. The separate testing set should be foreign to the machine learning algorithm and the purpose of it is to test if what it learned can really go out and do useful things accurately(e.g. making predictions on data). The researchers would have to make sure the testing data contains no data that the program has seen previously in the training phase, as…
