It is refer as “dead neurons”, when we train a neural network improperly, as a result, some neurons die, produce unchangeable activation and never revive.
Sometime when we train our model, we not only want to monitor some standard metrics (e.g. loss, accuracy, MSE) but also want to put in a short script to probe the model once in a while. For example, visualization…