Understanding the math behind Neural Networks

Neural Networks (NNs) are the typical algorithms employed in deep learning tasks. The reason why they are so popular is, intuitively, because of their ‘deep’ understanding of data, which is provided thanks to their peculiar structure. NNs, indeed, are built in the same way as the human brain’s neurons. Further, they aim at mimicking the way those networks send and receive impulses — basically, NNs mimic the way the human brain actually works.

Another interesting property of NNs is their being flexible in terms of structure and complexity: as you will see soon, they need only few, basic elements to work: all the exceeding stuff is something that might make them work better, but the inner structure remains the same.

In this article, I will explain the math behind a simple NN, building it from scratch without any language programming, only armed with matrices’ calculations (do not panic: data will be extremely simple and easy to handle). Of course, useful and powerful NN are rather built with Tensorflow, Keras or Pytorch, nevertheless, you will see that, by understanding the idea behind, you will find far easier to build your algorithm with code.

So let’s start. As anticipated, NNs replicates the structure of our biological neurons, and they look like that: