# Hierarchy of Vector Space Structures **(1) Raw Vector Space** This is a set of vectors with operations: addition and scalar multiplication. No concept of length, angle, orthogonality or direction. Think of this like a big *bag* of vectors, with no structure. At this point *picturing* vectors as arrows is *smuggling* in *extra structure* (geometry, length, direction). The only structure present is: 1. If you take two vectors and add them, you get another vector in the bag $u, v \in V \text{, then } u + v \in V$ 2. If you scale a vector, you get another vector in the bag $\text{If } a \in \mathbb{F} \text{ (the field of scalars) and } u \in V \text{, then } \; au \in V$ 3. There is a special zero vector that acts like adding nothing $\text{There exists } 0 \in V \text{ such that } u + 0 = u \text{ for all } u \in V$ **(2) Geometric Structure: Inner Product** Introduce an inner product to get length, angle and orthogonality. Still, there is no choice of "east"—all directions are equally valid. But we have made a *choice*: selecting an inner product. Technically, we are taking a vector space $(V, +, \cdot)$ over a field $\mathbb{F}$ (usually $\mathbb{R}$), and adding a new structure: the inner product. This is a function: $\langle \cdot, \cdot \rangle : V \times V \to \mathbb{F}$ such that for all $u, v, w \in V$ and scalars $a \in \mathbb{F}$: 1. There is linearity in the first argument (over $\mathbb{R}$): $\langle a u + v, w \rangle = a \langle u, w \rangle + \langle v, w \rangle$ 2. Symmetry (over $\mathbb{R}$): $\langle u, v \rangle = \langle v, u \rangle$ 3. Positive-definitene: $\langle v, v \rangle \ge 0 \quad\text{and}\quad \langle v, v \rangle = 0 \ \Leftrightarrow\ v = 0$ These three things allow us to define critical, new geometric concepts. Once we have $\langle \cdot, \cdot \rangle$, we can define: * Norm (length). Now you can say “u is longer than v” or “u is a unit vector”: $\|v\| := \sqrt{\langle v, v \rangle}$ * Distance between vectors. Now you have a geometry — vectors have locations relative to each other: $d(u, v) := \|u - v\|$ * Angle between vectors. Now you can measure orientation between vectors: $\text{For }u, v \neq 0: \cos \theta = \frac{\langle u, v \rangle}{\|u\| \, \|v\|}$ * Orthogonality: $u \perp v \iff \langle u, v \rangle = 0$ **(3) Pick A First Axis (Reference Direction)** Now we pick an arbitrary reference direction in the vector space. Call this $e_1$ (this is almost always a unit vector). We may refer to this reference direction as saying we have chosen an "east" direction. Then the direction orthogonal to it may be "north". And so on. In just a raw vector space we can't even formulate a notion of direction. We can only do so once we have an inner product. **(4) Iteratively Construct An Orthonormal Basis** From this, we then iteratively pick orthogonal vectors, $e_2, e_3, \dots$ **(5) Declare This Basis To Be Standard** Finally, we say that this is the standard basis. # Confusion 1: Coordinates are coefficients, not geometric lengths Suppose I pick a basis of $v_1$ and $v_2$: $v_1 = \begin{bmatrix}2\\0\end{bmatrix}, \;\; v_2 = \begin{bmatrix}0\\2\end{bmatrix}$ In standard euclidean geometry, $v_1$ has a length of $2$. Any vector I represent in this basis is really a linear combination of the basis vectors. Say I have a vector $d$: $d = \begin{bmatrix}1\\0\end{bmatrix} = 1v_1 + 0v_2$ If we just compute the length of $d$ via a euclidean distance calculation, we'll see that $||d|| = 1$. But of course that is not the *true, intrinsic* length of $d$. That is clearly $2$. If we want to arrive at the correct intrinsic length, we must perform a conversion via the **Gram Matrix**. # Confusion 2: Pure Math vs Physics/Applied Math **Pure Math** Part of the confusion here comes from failing to distinguish between pure and applied mathematics. In pure mathematics you start with $\mathbb{R}^n$, which is defined as: $\mathbb{R}^n := \{ (x_1, x_2, \dots, x_n) \mid x_i \in \mathbb{R} \}$ With addition and scalar multiplication clearly defined. Here, the *standard basis* is *baked in* from the definition: they are literally elements of the set! $e_1 = (1, 0, \dots, 0) , \;\; e_2 = (0, 1, \dots, 0)$ This is possible because $\mathbb{R}^n$ is already given as *coordinates*. The "coordinates" *are* the elements. In this view $\mathbb{R}^n$ already *is* a coordinate system — so a standard basis is given by definition. At this point notice that $\mathbb{R}^n$ is *not* a bunch of arrows. It is simply [Isomorphic](Isomorphism.md) to these arrows ([Elements of Rn as Vectors](Elements%20of%20Rn%20as%20Vectors.md), [All Vectors Spaces are Isomorphic to Rn](All%20Vectors%20Spaces%20are%20Isomorphic%20to%20Rn.md)). **Applied Math** Now consider physical space. This is just the set of all positions—there is no axes, origin or preferred direction. To describe vectors we must: 1. Pick an origin 2. Pick mutually orthogonal unit vectors — the "axes" 3. Assign coordinates relative to this choice Nothing in the physical space tells us which direction is $e_1$—we have to *choose*. Suppose we chose $m$ to be our first basis vector—called $e_1$. We the pick additional vectors as $e_2$, $e_3$ and so on. At this point we do not have coordinates. Just a reference frame. *Now* we define coordinates via the mapping: $e_1 \mapsto [1,0,0,\dots], \quad e_2 \mapsto [0,1,0,\dots], \ \dots$ And so saying $m$ is $e_1$ is a geometric *choice*. Saying $m$ has coordinates $[1,0,0,\dots]$ is a *coordinate representation*—which only makes sense after the whole basis is fixed and the coordinate map is defined. **Data Science** The same thing applies for data science. Suppose we had a data matrix $X$, with rows that are points in time and columns that represent prices at certain locations. This data matrix can be thought of a *specific description of price configuration space with respect to a particular basis*. Here there is no canonical, standard basis. So whatever the original basis for $X$ is, we can never fully escape it. Even if we change our basis, our new basis will be expressed relative—with respect to—the basis of $X$. --- Date: 20250811 Links to: Tags: References: * []()