Numerical Mathematics for Data Scientists: Part 1
This article series is dedicated to Numerical Mathematics which is inspired from courses MIT 18.06 — Linear Algebra and EAS 501 — Numerical Mathematics for Data Scientists that is being taught to me here at UB this semester. I’ll be sharing short yet explanatory notes on the various topics taught in these course and try to give my verdict on how they could be useful for Data Scientists.
Hey there! I am an MS student at UB focusing in Data Science and I’ve come here to share my learnings from various courses that I’ll be studying in this remarkable journey.
Feel free to drop your comments along the way and I’ll try to be consistent in replying to them. Without further ado, let us begin.
Vectors
A vector is an organized collection of numbers (also known as components). Also known as tuples, a vector’s length is determined by the number of components in it.
It is important to note that although the lengths of vectors a and b are the same, i.e. 3, the vectors a and b are different vectors. For two vectors to be equal to each other, the vectors must have all their components to be the same.
Vectors are of two types, namely, row vectors and column vectors. Although it is very clear by the nomenclature, yet to be clear, both vectors a and b are column vectors.
The dimensions of a vector are m x n, where m represents the number of rows and n represents the number of columns. Row vectors are of the form 1 x n whereas column vectors are of the form m x 1.
As a vector is a collection of numbers is it often described as a mathematical construct that contains information about both magnitude and direction.
Vector Operations
Let us now see how vectors can interact which each other.
The later line is read as “where u & v belong to the space R³” which means that u & v belong to the 3-dimensional vector space.
- Addition
2. Subtraction
3. Scalar Multiplication
A scalar is a vector in one-dimension, which means it is a vector with just one component.
4. Associativity
5. Transpose
Transpose is an operation that flips a vector by 90⁰.
While we are at the operations of vectors, it will be good to mention two special vectors: Zero Vector and One Vector.
Dot Product
The dot product is a vector multiplication operation performed on two vectors of the same dimensions resulting in a scalar.
Vector Norm
A vector norm is a vector operation that provides us with an estimation of the magnitude of a vector or “how big is the vector”.
Let us look at u.u
The above expression looks awfully like the euclidean distance, or simply, the distance of a point in n-dimensional vector space from the origin. This is used to define the 2-norm of a vector u.
A unit vector is a vector with 2-norm equal to 1. If a vector v is not a unit vector than a unit vector u in the same direction as v can be calculated using —
In general, the p-norm of a vector is defined as:
Typically, 1-norm is just the summation of all the components of a vector and is often called Taxicab Distance or Manhattan Distance. On the other hand, infinity-norm is a totally different concept.
Geometric Interpretation of Inner Product
Let u and v be two vectors in the same vector space, and neither of them is a zero vector. If u.v = 0, then the two vectors are called orthogonal. In fact, u.v = 0 if and only if u and v are orthogonal.
Cauchy-Schwartz Inequality
Triangle Inequality
The equation in red is called The Triangle Inequality and it holds true for all p-norms.
Linear Combination of Vectors
Linear Combination is simply the weighted sum of vectors. Let u and v be two vectors in the same dimension, and a and b be scalars. w = a.u + b.v is a linear combination of u and v.
Let us look at some identities of Linear Combinations.
To round up, vectors are the very building blocks of Algebra and the understanding of operations on vectors has the potential to rescue Data Scientists in tricky situations. In the next article, I’ll talk about Matrices and their operations.
Please forgive my crude penmanship.
