Linear Algebra: The Brilliant Building Block of Machine Learning

A bird’s eye view of Linear Algebra concepts for complete beginners, including readers from non-Computer Science and non-Mathematics backgrounds

The future is here. Like it or not, Machine Learning (ML) is going to have a huge say in defining this common future of ours. The ability to learn is undoubtedly one of the distinct qualities of human beings, and ML is that area of Computer Science that promises to give this unique ability to our machines and software.

The mathematics of Machine Learning includes a variety of different areas. So you might wonder, why Linear Algebra? Well, here is why:

Linear Algebra features among the best topics one can rely on, in order to get a good picture of ML.

So without much ado, let us get our feet wet.

Who doesn’t know that data is the new oil? It is from huge volumes of data, that computers are able to derive patterns and learn relationships- the founding principle of Machine Learning and Deep Learning.

So this calls for organizing data in formats that can easily be understood and processed by our machines and software. Unlike us human beings, machines understand only if we constantly feed data in a consistent way each time.

This is exactly why the notion of structured data came up. Linear Algebra was the first to introduce such mathematical objects to realize the need for structuring our data.

Some of the most commonly used mathematical objects:

A scalar is just a number, like 24.

A vector is an ordered array of values that can be stored as either a row or a column. A vector is always one-dimensional (row or column). We refer to a value inside the vector with its index. For example, the 3rd value in the above vectors is 7. The size of the both vectors above, is 1x1(1 row and 1 column)

A matrix is a vector expanded along both the row and column dimensions. Matrices are usually two-dimensional (row x column). To refer to any element in the matrix, we refer with 2 indices. The first index specifies which row the element belongs to. The second index specifies which column the element belongs to. For example, at (1,2) we have the element 4(which is located at the meeting of the 1st row and 2nd column). The size of the above matrix is 2x3(2 rows and 3 columns).

If I were to give one term for all these structured mathematical objects, I would call it a tensor. A tensor can be an n-dimensional array, and anything from a vector(1st order tensor-one dimension) to a matrix(2nd order tensor-two dimensions) or beyond(three dimensions or more).

So, that was a brief description of how Linear Algebra defines a good ‘language for data’. Just as we have defined the above conventions for how to store our data in a standard format, Linear Algebra also clarifies on the rules for computations (addition, subtraction, multiplication, division and more) with these data structures. Since these are entire sets of numbers we are playing with, Linear Algebra clearly teaches us how to manipulate these structures and derive key results from these operations.

Now, let us talk about exactly where these ideas come into play in the field of Machine Learning.

This is a snippet from a popular dataset used in ML, the Iris flowers dataset. The dataset contains a set of 150 records under five attributes — petal length, petal width, sepal length, sepal width and species. Here, each record/row represents one observation and each of the five values in a row, correspond to the five attributes mentioned above. This kind of data is exactly the kind we require for training our ML models to learn- the linear algebra format of the 2-D matrix.

But sometimes, data doesn’t exactly come ready-made in this numeric tensor format that we require it in, to give to our software. While several datasets are available in matrix format, not all are. Data can be images or photographs. You can think of the image as a rectangle with a width and height, and one pixel value in each cell for black and white images or 3 pixel values in each cell for a colour image. A photo is yet another example of a matrix from Linear Algebra. Operations on the image, such as cropping, scaling, shearing, and so on are all described using the notation and operations of linear algebra.

Suppose, you want to represent categorical data so you can train your classifier to recognize correctly. For example, you have 2 categories- Dog and Cat. If you wanted to specify a set of images in terms of what animal they represent, you could do it this way:

So the first image we have, is that of a dog(1 under ‘dog’ category). The second image, is that of a cat. The third one is of a dog. This is actually a popular method called ‘one-hot encoding’, because the occurrence of the one is what determines which category your image belongs to.

Similarly, the method of Latent Semantic Indexing used in Natural Language Processing represents entire documents as large matrices of word occurrences. Here, the rows could represent sentences, paragraphs or pages and the columns represent the words in the vocabulary of the text. Each cell in this matrix usually contains the count of the corresponding word in that sentence/para/page. Often, these matrices would be sparse i.e., several entries would be zero(representing non-occurrence of those words). These zero values are usually of little consequence and are safely filtered out, to retain only the essence of the document through methods like Singular Value Decomposition(SVD). Documents processed in this way are much easier to compare, query, and use as the basis for a supervised machine learning model.

Linear algebra is central to the description of deep learning methods via matrix notation to the implementation of deep learning methods such as Google’s TensorFlow Python library that has the word “tensor” in its name.

I hope that this article has given you a good intuition of how Linear Algebra helps with the representation and manipulation of data, specifically for Machine Learning.

If you enjoyed reading this article, please click the ‘clap’ icon. Thank you for reading this article and hope it has inspired you to continue reading on this topic.