Neural Networks From Scratch Series
The Mathematics Behind Neural Networks
This post describes a series of articles I started publishing on Medium which focuses on the topic of neural networks. More specifically, the series focuses on deriving the mathematics behind neural networks from scratch.
I took on this project because I wasn’t satisfied with the content on neural networks that I could find online at the time when I became interested in the field myself. I found that there was a deficiency in the mathematical foundation of neural networks in online articles for those readers who didn’t necessarily want to read academic papers on the topic or take a full course but still wanted to get a somewhat reasonable understanding of the math involved (although not in all detail).
While some of the articles in the series are intended to be aimed at any reader who is interested in learning about neural networks, others expect the reader to be familiar with some level of calculus. The articles present graduate level course material on neural networks explained in the simplest terms possible while involving the most relevant (but not complete) mathematical derivations. The series is meant to be read in order of the articles I provide here because the content of every article is built on top of the content of the preceding articles.
This is a live document so I will continue to add the embedded links to the articles in the series as I continue to publish them in the future.
This being said, I hope this series hits that sweet balance between concept and math for my readers and I hope you enjoy it!
- The Perceptron — The Building Block of Neural Networks
This is the first article in the series and it is friendly for all types of readers regardless of their math background (some minimal high school level math…maybe). The article introduces the perceptron and how it can be used to compute the logical ‘OR’ statement.
2. Training a Single Perceptron
This is the second article in the series which discusses a single perceptron model and what it means for a perceptron to learn. Some calculus experience is recommended here as the post derives the basic math behind the learning algorithm of the perceptron.
3. How Neural Networks Learn
This article builds on the mathematics introduced in the second article of the series — 2. Training a Single Perceptron. The math gets a bit more involved as the derivation of the Feedforward Backpropagation Algorithm is discussed here. Basic to intermediate level of calculus is expected.
