MACHINE LEARNING ALGEBRA
What Is the Difference Between a Tensor, a Matrix, and a Vector?
Read this short guide for a description of the difference between a tensor, matrix, and vector
Algebra is an important element of mathematics and has a lot of practical applications. Among other things, it plays a crucial role in the economy, quantum computing, and machine learning. For the latter one, matrices and vectors are important, while the popular Python framework PyTorch uses tensor-based operations. Despite their similarities, a tensor is neither a matrix nor a vector, contrary to what many people think.
A matrix is a grid of m x n numbers surrounded by square brackets. Here, m is the number of rows and n is the number of columns. Mathematical operations can be performed on matrices, such as e.g. matrix multiplication, matrix addition, and many more.
A vector is a 1D array of numbers, a matrix where m or n is equal to 1. Similarly to a matrix, it is also possible to perform numerous mathematical operations on a vector, and it is possible to multiply matrices with vectors and vice versa.
A tensor, however, can be thought of as a generalized matrix which can be described by its rank. The rank of a tensor is an integer number of 0 or higher. A tensor with rank 0 can be represented by a scalar, a tensor with rank 1 can be represented by a vector and a tensor of rank 2 can be represented by a matrix. There are also tensors of rank 3 and higher, the latter ones being more difficult to visualize. In addition to the rank, there are certain characteristics of tensors related to how they interact with other mathematical entities. If one of the entities in an interaction transform the other entity or entities, then the tensor has to obey a related transformation rule.
[1] Steven Steinke. What’s the difference between a matrix and a tensor? (Aug 2017). https://medium.com/@quantumsteinke/whats-the-difference-between-a-matrix-and-a-tensor-4505fbdc576c
