Machine Learning Basics : Scalars, Vectors, Matrices and Tensors

Machine Learning involves several types of mathematical objects:

Scalar

A scalar is just a single number, in contrast to most of the other objects like Vectors, which are usually arrays of multiple numbers.

We write scalars in italics. We usually give scalars lower-case variable names. When we introduce them, we specify what kind of number they are.

For example,

we might say “Let s ∈ R be the slope of the line,” while defining a real-valued scalar, or

“Let n ∈ N be the number of units,” while defining a natural number scalar.

Vectors

A vector is an array of numbers. The numbers are arranged in order. We can identify each individual number by its index in that ordering.

Typically we give vectors lower case names written in bold typeface, such as x.

The elements of the vector are identified by writing its name in italic typeface, with a subscript. The first element of x is x1, the second element is x2 and so on.

We also need to say what kind of numbers are stored in the vector. If each element is in R, and the vector has n elements, then the vector lies in the set formed by taking the Cartesian product of R n times, denoted as Rn.

When we need to explicitly identify the elements of a vector, we write them as a column enclosed in square brackets:

Matrices

A matrix is a 2-D array of numbers, so each element is identified by two indices instead of just one.

We usually give matrices upper-case variable names with bold typeface, such as A. If a real-valued matrix A has a height of m and a width of n, then we say that A ∈ Rm×n.

We usually identify the elements of a matrix using its name in italic but not bold font, and the indices are listed with separating commas.

Tensors

In some cases we will need an array with more than two axes.

In the general case, an array of numbers arranged on a regular grid with a variable number of axes is known as a tensor.

We denote a tensor named “A” with this typeface: A. We identify the element of A at coordinates (i, j, k) by writing Ai,j,k.