Difference between Training, Validation and Test Data Set.
While modelling and training a Dataset in Machine Learning, people often get confused regarding this concept of training, validation and test Data set.
Here, In this blog, I have simply explained what are the key differences among these terms.
Let’s imagine you have a magic pencil that can draw cool pictures. Now, you want to draw the best picture of a Phoenix. Here’s how you can use the magic pencil to make it perfect:
1. Practice Drawing at Home (Training):
First, you practice drawing at home. You use your magic pencil to draw lots of Phoenix until you become really good at it.
2. Show Your Drawings to Parents (Validation):
Next, you show your Phoenix drawings to your parents to get their opinions. They help you by providing feedback like how it can be improved (validation).
3. Create the Perfect Phoenix Drawing (Test):
After practicing and getting feedback, you want to make sure your Phoenix drawing is the best it can be. This is the final test to see if drawing is truly magical and works on any paper.
In this story:
Training (Home Practice): This is like your magic pencil learning from lots of examples (training data).
Validation (Showing Parents): They help you improve your drawings by giving feedback, just like a validation set helps improve a model during training.
Test (Creating the Perfect Drawing): The test is like drawing the final Phoenix on a new piece of paper that you’ve never used before. This shows how good your magic pencil has become at drawing Phoenix without anyone’s help.
