Hilbert Space
What is a Hilbert Space? This term pops up quite often in various domains like partial differential equations, quantum mechanics and machine learning. Most of us only have a sketchy understanding of hilbert spaces, thinking of it as having something to do with inner products. A geeky friend of yours may state the three golden properties associated with it, but is that enough?
In this blog, I aim to develop a comprehensive understanding of hilbert spaces cutting through the mathematical jargon. This doesn’t mean I won’t involve any maths. In fact, I aim to develop a complete mathematical understanding of hilbert space by getting the fundamentals straight. So let’s get started with our journey to land on Hilbert Space.
What is a Space?
Space in “Hilbert Space”, is a mathematical construct and not the “space” which we normally understand. Space in maths, signifies a collection of vectors, which interact in a certain manner. It has a defined set of operators (eg. addition, subtraction etc.), distance metric (function to measure the distance between two vectors) among others.
The space around us, is a subset of the Space we just defined, where 3D vectors follow euclidean geometry. Now we have some idea about the generic definition of space. We will get started with our initial goal of understanding hilbert space. We will follow the route specified in the diagram below, understanding every super set before going further.
Topological Space
Wait, how did we come to terms with the above structure first? Are space and topological space the same thing? No, they aren’t. Space is classified into two basic types linear spaces and topological spaces. This is a coarse classification, they aren’t mutually exclusive. There exists metric spaces which fall under both categories (don’t worry, we will come to this).
Linear spaces are vector spaces which have pre-defined operations which obey linearity. Linear spaces have certain limitations as we’re not able to define a perpendicular line or circle, as there’s no concept of an inner product.
Topological spaces are closer to the real world, and help us define things around us like uniform surfaces. Therefore, if we start from topological spaces our journey to reach Hilbert spaces would be easier.
What is a topology?
Before understanding topological space, it is important that we understand the meaning of topology. Keep in mind what you understand by the word “topology”, as we are going to define something similar mathematically.
In maths, topology is defined as a fundamental construct of space which is preserved under continuous deformation like stretching, folding, crumbling but not tearing (like your bed sheet). Mobius strip is a popular example of a topology shown below, which has a single surface and a singular boundary.
Enough talking, let the math kick in now. We define a topology on a non-empty-set X, as a collection of open sets (τ, a set of points which do not touch the boundary) in X, obeying the properties below
- empty set ϕ and X, are open
- Union of any possible combination of open sets is open
- Intersection of a finite number of open sets is open
Using the above definition, try to correlate all points that lie on the surface of the Mobius strip (shown above) except the boundary with X. You can also imagine all points lying inside a 2D circle (which doesn’t satisfy the boundary condition) as X. Now go over the three properties again. See, if it makes more sense intuitively.
Topological space is defined as the pair (X, τ), where τ is a topology over the set X. In situations where the definition of τ is clear from the context, X is directly refered to as the topological space. This is one of the ways to define topological space, other ways are mentioned here.
Metric Space
Metric Space is a subset of topological space which has a defined metric on the set. A metric is basically a function which computes the distance between two points in the space. The function outputs a real number d(x, y) to every pair x, y ∈ X. The function should satisfy the following properties
- d(x, y) ≥ 0, for all pairs (x, y) ∈ X
- d(x, y) = 0, if x = y
- d(x, y) = d(y, x), for all pairs (x, y) ∈ X [Commutative property]
- d(x, z) + d(z, y) ≥ d(x, y), for all (x, y, z) ∈ X [Triangle inequality]
These properties are intuitive enough as they hold true for the distance function we encounter in Euclidean space. Any function d satisfying these properties can be a metric function.
The metric space M is therefore defined as the ordered pair of (X, d). Some of the commonly used metric functions are shown below
Euclidean distance
Manhattan distance
Minkowski distance
Another important concept we need to touch upon to move forward is the “completeness” of a metric space. Now to understand this, we need to know about Cauchy sequences.
Cauchy Sequence is a sequence whose members come increasingly close to each other as the sequence progresses. For clarity, visualise the sequence of blue dots and how they become closer as n increases.
In the context of a metric space [M=(X, d)], cauchy sequence is defined as the sequence whose distance becomes smaller as the series proceeds. Such that, given m, n > N (a positive integer), the following holds true
Thus, the sequence converges to an element in X. A metric space is known as complete if every cauchy sequence in it, converges to some element in X.
Normed Vector Space
Metric spaces don’t have any operators defined on them apart from the metric function d. However, most of the applications arising in analysis are derived from a norm, denoting the length of the vector.
What is a norm?
Norm is a function which outputs a positive value denoting its length or magnitude to every vector in a vector space (except a zero vector). It usually denoted as ||x||.
The equation above is the generic definition of a norm. By assigning values in p, we see few known instances. For p=1, it is the absolute-value norm and p=2 it is the euclidean norm.
Coming back to metric spaces, now we’ve gathered the tools to define a normed vector space. A normed vector space is simply a metric space whose distance function is defined as the length of the difference vector.
Every normed space is both a linear topological space and a metric space. A perfect example of the intersection between linear and topological space.
Banach Space
Here we introduce a special instance of normed vector space. Banach Space is a complete normed linear vector space. Banach space obeys the completeness property arising due to cauchy sequences.
Inner Product Space
Inner product spaces introduce a new structure known as the inner product. It is the same as dot product with scalars in Euclidean space. It is a simple concept, but I would still like to reiterate its basic principles for clarity
What is an inner product?
It is a mathematical operator which acts on two vectors to produce a scalar quantity (depending on which field the vectors are define). It obeys the following properties
- Linearity
- Conjugate Symmetry
- Postive - definiteness
Inner products add an important structure to metric spaces, as orthogonality and angle between vectors can now be computed. If we break down inner products, it comes down to euclidean norm.
Therefore, it is a type of normed vector space (it doesn’t satisfy completeness though). Inner product spaces are a great way to generalise euclidean spaces. As it is easy to extrapolate the basic principles to higher dimensional vectors.
Hilbert Space
We have arrived at our final destination. We’ve already covered so much ground that there’s hardly anything left to add.
Hilbert Space is an inner product space which satisfies completeness. The additional structure that makes it complete is the introduction of a defined distance function
Now, we’ve successfully landed on Hilbert space. The journey may have been abrupt so go back and revisit the path we traversed to reach here. Dive deep into any of the aspects you feel hazy (I’ve tried to provide quite a few links for reference in the text).
Notes on Hilbert
David Hilbert was one of the greatest mathematician of the 19th and 20th century. He was a child prodigy and received his PhD at 23. He worked with Einstein to mathematically formulate equations of general relativity (for which he claimed no credit). Hilbert defined his 23 famous maths problem which shook the world (including Riemann Hypothesis). Hilbert retired from professorship at the age of 68, and had to spend his remaining days in Nazi Germany. Here’s one of his famous quotes
Wir mussen wissen. Wir werden wissen. (We must know. We will know.)
-David Hilbert
