# Four Fundamental Subspaces The **Four Fundamental Subspaces** are a way to divide up the input and output space of a linear transformation. We have the parts that are *transformed* (in red below) and then the parts that are *thrown away* (either squished to $0$, i.e. the [Null Space](Null%20Space.md), or are orthogonal to what can be output, the null space of the transpose).  # Concrete Example To make this concrete, consider a linear transformation $A$ that acts as a map between $\mathbb{R}^2$ and $\mathbb{R}^3$:  Because it is a mapping between two and three dimensions, it can be represented by a 2x3 matrix:  ### Fundamental Subspace 1: The Range of $A$ Let's further say that the columns of this matrix are [linearly dependent](Linear%20Dependence.md), meaning that the [range of $A$ (or the column space)](Column%20Space.md) is a line in $\mathbb{R}^3$: .png) Remember, the [Column Space](Column%20Space.md) of $A$ is the set of all vectors that the can be reached via a [Linear Combination](Linear%20Combination.md) of the columns of $A$: $R(A) = \big\{ Ax \mid x \in \mathbb{R}^2 \big\}$ Put another way, it is the set of all vectors that $A$ maps to in the output space. So, we take any vector in $\mathbb{R}^2$, apply $A$ to it (which involves taking a linear combination of the columns of the matrix representing $A$), and you get a vector whose tip is on the red line. ### Fundamental Subspace 2: The Null Space of $A$ Let's also say that the [Null Space](Null%20Space.md) of $A$ is a line, specifically the blue line below:  Again, for review, the null space of $A$ is defined as the set of all vectors that $A$ maps to the zero vector: $N(A) = \big\{ x \in \mathbb{R}^2 \mid Ax = 0 \big\}$ Visually, just to hammer this home, this looks like:  ### Fundamental Subspace 3: Range of $A^T$ Because we know that the null space will end up at 0, applying $A$ will squish space inwards as shown below by the blue arrows. This means all the vectors in $\mathbb{R}^2$ will end up on the red line:  This line is *orthogonal* to the null space of $A$. The line will then be thrown into $\mathbb{R}^3$ and it will end up as the range of $A$. Now, this line in $\mathbb{R}^2$ happens to be the range of $A^T$!  As a preview, it is worth highlighting that the range/column space of $A^T$ is actually just the [Row Space](Row%20Space.md) of $A$ (see [here](https://math.stackexchange.com/questions/3516169/is-row-space-of-a-matrix-equal-to-the-column-space-of-the-transpose-of-matrix))! However, that isn't clear from anything that we have seen yet, so for now we will say it is the range of some other map, which we will call $?$. Leaving off here: https://www.youtube.com/watch?v=yfj8uMwAgrI&t=216s * Good line at [this part of video](https://youtu.be/yfj8uMwAgrI?t=340): * The key takeaway is that the transformation, $A$, and it's pseudoinverse $A^+$, provide a way to translate from the *same line* (i.e. the *subspace*) between the input space and output space. * Another way to look at this is taht the null space of $A^+$ gives a sense of what vectors in the output space can't be mapped to by $A$. And the null space of $A$ gives a sense of what can't be mapped to be $A^+$. * The transpose can be thought of as a pseudo inverse of sorts * The tranpose allows us to translate an important space, in this case the range of A, between R2 and R3 * The two ranges are not the same line of course; rather, they are linear transformations of each other. We can see $A$ as performing some kind of dimension maintaining transformation (this just means the line stays a line) on just the range of $A^+$ to arrive a different subspace (i.e. it is now embedded in R3 and not R2). * So, the four fundamental subspaces allow us a nice way of describing which subspaces are transformed (the ranges, red lines), and which are thrown out, the null spaces. * In SVD the transpose comes in because we are trying to reverse (inverse) our original transformation. Because U and V are orthogonal, the transpose and inverse are equivalent. --- Date: 20220105 Links to: [Linear Algebra MOC](Linear%20Algebra%20MOC.md) [Singular Value Decomposition](Singular%20Value%20Decomposition.md) Tags: References: * [# Matrix Transpose and the Four Fundamental Subspaces (Fantastic Video)](youtube.com/watch?v=yfj8uMwAgrI&t=634s)