# Inner Product In linear algebra, if we are only allowed to use the concept of a **subspace** and **transformations**, then we have not notion of ***length***. But wait, you may say, if we are talking about *geometric vectors* we frequently discuss length, orthogonality and angles. And this is indeed true *specifically for geometric vectors*. However, if we are talking about elements of $\mathbb{R}^n$ we cannot talk about length given only the concepts of subspaces and transformations. The reason is that sets of numbers, i.e. n-tuples, are just that: sets of numbers. They have no notion of length. You cannot take a tape measure and measure them (as you can with geometric vectors). In order to bring in the concept of *length* (which is so central to virtually all of applied mathematics) from just geometric vectors to *all other types of vectors* (such as $\mathbb{R}^n$ and polynomials) we need new mathematical tools. You may ask: Why is it so important to bring in the concept of length? One reason is that we have to often discuss proximity! How close is one solution to another? In a sense large parts of mathematics are about relationships between objects, so it makes sense that notions of length arise! You may ask, well surely we do have a concept of length! Simply have the square root of sum of squares as we do with geometric vectors. While this is not wrong, it should be clear that it is a *choice* we make; it is not a rule of law we must obey. Now, this is not incorrect. But there are many *other correct answers*! And some answers may indeed be more useful in certain contexts. So, the key idea here is: > There are many things that we can **define as a length**. (see [Geometry](Geometry.md)) The question remains: should we simply use the square root of sum of squares as length ($L_2$)? There is a very strong case to be made that the answer is *no*! This is because we want our length to apply to *all vector spaces*, not simply geometric vectors or $\mathbb{R}^n$. How would $L_2$ apply to polynomials? It is not directly clear. The approach that mathematicians have taken is: ***to not define length directly***. Attempts were made an abandoned. The way to introduce length to various spaces is through the concept of **inner products** (for geometric vectors these are called scalar products). The inner product will be revealed to be much more fundamental than length. We will also see that things like length and angles follow from the inner product. Note that this is very similar to the notion expressed by Jordan Ellenberg in [Shape](Shape.md); namely: To assign a set of **objects** a **[Geometry](Geometry.md)** is, at bottom, to assign a number to any two objects in that set, which we interpret as **distance**. The inner product accomplishes this exactly. ### Inner Product vs. Dot Product We can see that the dot product is so useful in geometry (geometric vectors) that we would like to have something like it for other vector spaces. That is what the inner product is! The inner product is a **generalization** of the dot product for arbitrary vector spaces. ### In the context of functions Essentially think of the dot product applied to functions. The intuition is outlined incredibly nicely in [Abstract Vector Spaces](Abstract%20Vector%20Spaces.md) and the associated 3b1b video. In mathematics, an inner product space is a real vector space or a complex vector space with a binary operation called an inner product. The inner product of two vectors in the space is a scalar, often denoted with angle brackets, as in $\langle a,b\rangle$. Inner products allow formal definitions of intuitive geometric notions, such as lengths, angles, and orthogonality (zero inner product) of vectors. Inner product spaces generalize Euclidean vector spaces, in which the inner product is the dot product or scalar product of Cartesian coordinates.[3] Note that a vector space with lengths and angles is an inner product space. ### Inner product *defines* angles Edit, for the issue of orthogonality. The pressing issue here is that inner products define what it means to be orthogonal. So I issue a challenge to you here. Without referencing an inner product (this includes angles, as the inner product DEFINES angles), what does it mean to be orthogonal? You can't answer this question. The entire notion of angle, orthogonality, etc. are summarized in: ### Inner Products, Norms and Metrics They form a hierarchy, which is nicely defined [here](https://www.youtube.com/watch?v=VGidbLaddVY&list=PLAJOFd5cEXsrNcvSgEDgRSZ0G6mQOtCIT&index=8&t=600s). --- References: * [Why Inner Products? - YouTube](https://www.youtube.com/watch?v=Ww_aQqWZhz8&t=896s) * [Wikipedia](https://en.wikipedia.org/wiki/Inner_product_space) * [Bras and Kets I: Standard Inner Products](https://www.youtube.com/watch?v=_rkuNoL3q3M) * [Inner products, Steve Brunton](https://www.youtube.com/watch?v=g-eNeXlZKAQ&t=1s) * [What is the geometric meaning of the inner product of two functions? - Mathematics Stack Exchange](https://math.stackexchange.com/questions/1414389/what-is-the-geometric-meaning-of-the-inner-product-of-two-functions)