ML-Neural Networks
NN’s cost function-Andrew Ng
Like logistic regression, neural network also has a cost function. Before looking at the actual function in mathematic notations, we have to realize that we only saw a NN that had just one output. However, there are NNs that have multi outputs, for example multi-class classification.
Suppose we have a well trained NN that distinguishes objects in a road.
It’s basically a “one vs all” algorithm. If you look at the outputs of h(x), you can see that only one output node lights up (making it nearly 1) and others are set to 0.
Cost function
To recap, this was our cost function for logistic regression.
Easily saying, cost function for NN is a more generalized version of logistic regression. Let’s see it using the example that has K output units(K=4) given above with it’s notations.
Looks very complicated, but it’s just more summations due to the increase in output units.
- the double sum simply adds up the logistic regression costs calculated for each cell in the output layer; and
- the triple sum simply adds up the squares of all the individual Θs in the entire network.
- the i in the triple sum does not refer to training example i
