DEEP LEARNING

Understand Vectorization for Deep Learning

Compared to a “for” loop, NumPy in Python is over 10,000% faster when dealing with a billion multiplications.

Photo by Markus Krisetya on Unsplash

This post will introduce you to vectorization, and its importance in machine learning (especially deep learning). You will also learn how it is implemented in Python with NumPy, and how much of a difference it makes.

This post assumes a minimal background in linear algebra. You should know what a vector is, what a matrix is, and how we multiply vectors and matrices. Apart from these, I will introduce everything else as we go along.

Why is Vectorization Important?

When it comes to applying machine learning to a given problem, there is typically no deterministic formula that can tell you the precise architecture or hyperparameter that will do well. Regardless of what anyone claims, the procedure is iterative through trial and error. It is, by nature, empirical.

Adapted from Andrew Ng’s Deep Learning Course (Image by Author)

This is why we need to divide our datasets into different parts, a training set, a development set…