I have been confused with these two terms for a long time. They are very common words in machine learning and many said they are the same.

Today, I found a clear answer and they are slightly different. Have a look if you are concerned about it.

Loss Function:

measures how close the prediction is to the observed output, in a data point.

Cost Function:

the average loss over the training data. It is based on the loss function you chose.

When a model is learned, we are finding the parameter values that minimize the cost:

Originally published at http://github.com.