What is epoch and How to choose the correct number of epoch

At the output layer of the neural network, we have a flexible decision boundary at each output node.

As these are flexible in nature, we try to make them get the shape of the data. These boundaries are the combination of various lines in the hidden layer.

So for changing the shape of boundary, we change the weights these lines of hidden units.

The number of epoch will decide- how many times we will change the weights of the network.

As the number of epochs increases, the same number of times weights are changed in the neural network and the boundary goes from underfitting to optimal to overfitting.

So, what is the right number of epochs?

Well, the correct answer is the number of epochs is not that significant. more important is the validation and training error. As long as these two error keeps dropping, training should continue.

For instance, if the validation error starts increasing that might be an indication of overfitting.

You should set the number of epochs as high as possible and terminate the training when validation error start increasing