Loss Functions in Deep Learning

A Guide on the Concept of Loss Functions in Deep Learning — What they are, Why we need them…

Artem Oppermann
Mar 7 · 10 min read

1. Why do we need Loss Functions in Deep Learning?

Recap: Forward Propagation

Fig, 1. Feedforward neural network. Source: Author’s own image.
Eq.1 Forward propagation. Source: Author’s own image.
Eq. 2 Quadratic loss function. Source: Authors own image.

Minimizing the loss function automatically causes the neural network model to make better predictions regardless of the exact characteristics of the task at hand.

2. Mean Squared Error Loss Function

Eq. 3 MSE Loss Function. Source: Author’s own image.

When should You use Mean Squared Error loss?

3. Cross-Entropy Loss Function

Fig. 2 One-hot-encoded vector (left), prediction vector (right). Source: Authors own image.
Eq. 4 Cross-entropy loss function. Source: Author’s own image.
Fig. 3. Cross-Entropy function depending on prediction value. Source: Authors own image.

When should You use Cross-Entropy loss?

You should always use cross-entropy loss if probabilities are involved. Meaning if you are doing some kind of classification.

4. Mean Absolute Percentage Error

Source: Authors own image.
Source: Authors own image.

When should you use MAPE?

Take-Home-Message

MLearning.ai

Data Scientists must think like an artist when finding a solution

Sign up for AI & ART

By MLearning.ai

A weekly collection of the best news and resources on AI & ART Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Artem Oppermann

Written by

Deep Learning & AI Software Developer | MSc. Physics | Educator|

MLearning.ai

Data Scientists must think like an artist when finding a solution, when creating a piece of code.Artists enjoy working on interesting problems, even if there is no obvious answer.

Artem Oppermann

Written by

Deep Learning & AI Software Developer | MSc. Physics | Educator|

MLearning.ai

Data Scientists must think like an artist when finding a solution, when creating a piece of code.Artists enjoy working on interesting problems, even if there is no obvious answer.