# Spectral Decomposition **Spectral decomposition** is the process of *decomposing* a linear transformation into 3 parts: 1. A rotation 2. A scaling/reflection 3. A rotation Given a *symmetric* matrix $S$, spectral decomposition states that it can be decomposed into: $S = Q \Lambda Q^T$ Where $Q$ is an *orthogonal* matrix, $\Lambda$ is a *diagonal* matrix, and $Q^T$ is also orthogonal. If we think about $S$ as a function composition, we can think of it as: $\overbrace{\text{Rotation}}^Q \rightarrow \overbrace{\text{Stretch}}^{\Lambda} \rightarrow \overbrace{\text{Rotation}}^{Q^T} $ The decomposition works as follows:  We can state the following about the matrices making up the decomposition: * $Q$ *rotates* the standard basis (let us call the standard basis $u_1, u_2, \dots$ for this example) to align with the *eigenvectors* of $S$. We know this because the column vectors of a matrix represents where the basis vectors land after the transformation. * $Q^T$ would simply be a rotation in the reverse direction; i.e. the $i$th eigenvector will move to the $i$th standard basis. * $\Lambda$ will stretch via the eigenvalues of $A$ ### Visualization The best thing for you to do is watch this video [here](https://youtu.be/mhy-ZKSARxI?list=PLWhu9osGd2dB9uMG5gKBARmk73oHUUQZS&t=732). But I will add the most important frames below. The final transformation that $S$ produces is outlined in [diagram](Change%20of%20basis%20drawing.excalidraw.md). ### TLDR 1. Find the eigenvectors of $S$ 2. $Q^T$ will rotate the eigenvectors of $S$ to the standard basis (this rotation is a transformation, yielding a new space) 3. $\Lambda$ will scale the new space 4. $Q$ rotate backward from the standard basis to the eigenvectors A *very key idea* is that each of the 3 transformations above will be described with respect to the standard basis. In other words, we can think of the a grid (description) in the background at all times that our transformations are with respect to. For instance: 1. Our entire space is described with respect to the standard basis. $Q^T$ rotates the eigenvectors of $S$ (which are described wrt the standard basis) to the location of the standard basis. This rotation of the eigenvectors defines a full transformation (literally thats what $Q^T$ is) that rotates the entire space. 2. So our entire space was transformed. But we have the original space in the background that we describe all of our transformed vectors wrt! Think about this for a moment: the entire way that we *know* something was *transformed* is if we know where it was transformed *from*. That literally is what a transformation *is*; it is taking one thing and mapping it to another thing. [Visualize Spectral Decomposition | SEE Matrix, Chapter 2 - YouTube](https://youtu.be/mhy-ZKSARxI?list=PLWhu9osGd2dB9uMG5gKBARmk73oHUUQZS&t=587) ### Orthogonality Why is it so useful that symmetric matrices have eigenvectors that are orthogonal to each other? Well, that standard basis are also orthogonal! So, if we have orthogonal eigenvectors, then there must exist a *rotation* that rotates the eigenvectors to the standard basis, and vice verse. See more in this video [here](https://youtu.be/mhy-ZKSARxI?list=PLWhu9osGd2dB9uMG5gKBARmk73oHUUQZS&t=537). #### Relation to SVD How often do we actually get a symmetric matrix? This is where [Singular Value Decomposition](Singular%20Value%20Decomposition.md) comes in; it takes this idea of breaking down a matrix into a rotation, a stretch/reflect, and a rotation, and applies it to *any* matrix. --- Date: 20220714 Links to: Tags: #review References: * [Visualize Spectral Decomposition | SEE Matrix, Chapter 2 - YouTube](https://www.youtube.com/watch?v=mhy-ZKSARxI&list=PLWhu9osGd2dB9uMG5gKBARmk73oHUUQZS&index=4)