**From Scalar to Tensor: Fundamental Mathematics for Machine Learning with Intuitive Examples Part 1/3**

For understanding the mathematics for machine learning algorithms, especially deep learning algorithms, it is essential to building up the mathematical concepts from foundational to more advanced. Unfortunately, Mathematical theories are too hard/abstract/dry to digest in many cases. Imagine you are eating a pizza, it is always easier and more fun to go with a coke.

The purpose of this article is to **provide intuitive examples** for fundamental mathematical theories to make the learning experience more enjoyable and memorable, which is to serve chicken wings with beer, fries with ketchup, and rib-eye with wine.

The 3-course fundamental mathematics for machine learning meal is organized as follows:

**From Scalar to Tensor**: Fundamental Mathematics for Machine Learning with Intuitive Examples **Part 1/3**

- What are Scalar, Vector, Matrix, and Tensor?
- Addition between Scalar, Vector, and Matrix
- Multiplication between Scalar, Vector, and Matrix
- Identity and inverse matrix
- Diagonal matrix and symmetric matrix

- 1-Norm, 2-Norm, Max Norm of Vectors
- Orthogonal and orthonormal vectors
- Orthogonal matrix

- Eigendecomposition of matrix: eigenvalue and eigenvector
- The trace operator
- Determinant square matrix

In this article, we will go through the part 1/3, **From Scalar to Tensor **with intuitive examples.

**What are Scalar, Vector, Matrix and Tensor**

**Scalar** is a single number. Usually, we write scalars in *italic* and lowercase, such as “*x”*. Scalar could be any type of number, for example, natural number, rational number, or irrational number as shown below:

- Natural number: 3, 4, 5, …
- Rational number: 4/3, 1.666666…
- Irrational number:

The following code shows how to use Numpy’s N-dimensional array **ndarray**** **to create a scalar variable.

import numpy as np

x = np.array(1.666666)

**Vector** is an array of numbers. Typically, we denote vectors with **bold** and lowercase letter, and put elements of a vector as a column enclosed in square brackets:

The following code shows how to generate a vector with Numpy.

x = np.array([1,2,3,4,5,6,7,8,9])

**Matrix** is a 2-D array of numbers, which usually denoted with **bold** and uppercase variable names, such as:

The following code shows how to create a 4*4 matrix with Numpy.

x = np.array([[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]])

**Tensor** is a k-D array of numbers. The concept of tensor is a bit tricky. You can consider tensor as a container of numbers, and the container could be in any dimension. For example, scalars, vectors, and matrices, are considered as the simplest tensors:

- Scalar is a 0-dimensional tensor
- Vector is a 1-dimensional tensor
- Matrix is a 2-dimensional tensor

Thereby, we can deduce that a 3-D tensor is a cube, 4-D tensor is a vector of cubes, 5-D tensor is a matrix of cubes, 6-D tensor is a cube of cubes, etc.

We denote tensor in bold and uppercase letter, and identify the element of tensor at coordinates (*i, j, k*) by writing:

The following code shows how to create a 3-D tensor with Numpy.

x = np.array([[[1, 2, 3],[4, 5, 6],[7, 8, 9]],

[[10, 20, 30],[40, 50, 60],[70, 80, 90]],

[[100, 200, 300],[400, 500, 600],[700, 800, 900]]])

**In industry, commonly, we store time series data in 3-D tensor, image data in 4-D tensor, video data in 5-D tensor, etc.**

**Addition between Scalar, Vector, and Matrix**

**Scalar+Matrix** = add the scalar to each element in the matrix:

**Vector + Matrix** = add the vector to each row in the matrix:

**Matrix + Matrix** = add their corresponding elements as long as the matrices have the same shape:

#### Multiplication between Scalar, Vector, and Matrix

**Scalar・Matrix** = each element of the matrix multiply the scalar

**Matrix・Vector** = dot product of each row in the matrix and the vector

#### Identity and inverse matrix

In order to calculate the inverse of a matrix, we need to build the concept of identity matrix. Identity matrix is simple from structure point of view: all the entries along the main diagonal are 1, while all other entries are 0, such as:

The inverse of matrix **A **is denoted and defined as following:

The **necessary and sufficient condition** for the existence of the inverse of a matrix:

- the matrix must be
**square** - all the columns of the matrix must be
**linearly independent**

In the following example, the second column of the matrix could be considered as the first column multiply by 2, so that the matrix is linearly dependent, which means it does not have an inverse.

#### Diagonal matrix and symmetric matrix

A matrix is **diagonal** if and only if,

A square diagonal matrix whose diagonal entries are given by vector ** v **can be denoted as diag(

**), for example,**

*v*A symmetric matrix is any square matrix that is equal to its own transpose:

For example, the following 3*3 matrix is symmetric:

Congratulations! You just completed one-third of Fundamental Mathematics for Machine Learning with Intuitive Examples. It is straightforward to understand when there are examples, right?