Gradient of the 2-Norm

My blog: https://ai-research.dev/gradient-of-the-2-norm/

Prove L2 Norm of x = x^Tx when x is a vector:

And the properties of the transpose, we obtain:

Since A^TB is a scalar it equals its transpose: A^TB = (A^TB)^T = B^T(A^T)^T = B^TA So (1) can be simplified to

The gradient vector is a column vector containing the first-order partial derivatives

1. f(x) = c^Tx where c is a constant vector

2. f(x) = x^TBx where B is a symmetric matrix

• Matrix multiplication
• https://en.wikipedia.org/wiki/Matrix_multiplication

From (3), consider the k^th row in the above vector

Therefore,

From (2), we have

Using the formulas from the sections 1 & 2, with c=A^Tb & B = A^TA, we have