# Linear Algebra from first principles * A linear transformation tells you *exactly* how one [space](Space.md) is transformed into another space, where a certain set of properties (i.e. linearity) is preserved. * Linear algebra can be confusing because it so heavily relies upon our ability to distinguish between an *object* (vector) and it’s description/representation! ### Objects and representation Consider the vector $v$ below:  We can describe $v$ either with respect to a linear combination of $e_1$ and $e_2$, or with respect to a linear combination of $f_1$ and $f_2$: $ [v]_e = \begin{bmatrix} 2.5 \\ 2.5 \end{bmatrix} = 2.5 e_1 + 2.5 e_2 $ $ [v]_f = \begin{bmatrix} 1 \\ 1 \end{bmatrix} = 1 f_1 + 1 f_2 $ In both cases, the underlying $v$ remains the same! It is simply being described with a different linear combination of other vectors. So, we have two *distinct* things at play here: 1. The ***abstract mathematical object*** the **linear transformation**. This will transform vectors *regardless* of how they are *described* (regardless of what basis we use to transform them). If we recall the definition of [functions as sets](Space,%20Transformations%20and%20Descriptions.md) we can think of the set of pairs of vectors, where each vector in the domain is paired with a vector in the codomain (where this vector in the codomain is thought to be where the function mapped the vector in the domain). If we used this notion of a function, simply a set of pairs of vectors, we would have no need for a matrix/parameterization! However, that would be prohibitively expensive to work with in practice! It would also mean we miss out on being able to exploit rich structure *about the mapping itself*. 2. The specific ***instantiation*** of the linear transformation with respect to a particular basis. This is literally a **matrix**. This is unbelievably *crucial* to understand. A linear transformation is only defined a transforming from one space to another. There is no *basis* associated with it. When we pick a basis that it is dealing with, that is a parameterization, and that allows us to write out a matrix representing the transformation. So, again, at the risk of my voice becoming hoarse, a linear transformation simply transforms (linearly) mathematical objects (vectors in a vector space) from one space to another. It has no notion of coordinates. We provide coordinates, and hence instantiate a matrix representation, when we pick a basis with which we are going to represent all vectors with respect to. The reason it is so hard to break this association between the abstract function and the parameterization is because from an incredibly early age we are introduced to the cartesian coordinate system and built an incredibly strong association between objects and their *representation* (where they live with respect to some “fixed” coordinate system). At first you may simply think that is “the” coordinate system; after all, it is the main one that you have seen! Breaking this association is incredibly powerful in growing as an applied mathematician. This is outlined more in [A shortcoming of notation](Rotation%20and%20Change%20of%20Basis.md#A%20shortcoming%20of%20notation), as well as [Change of Basis and Matrices](Change%20of%20Basis%20Linear%20Algebra.md#Change%20of%20Basis%20and%20Matrices). Also checkout [Space, Transformations and Descriptions](Space,%20Transformations%20and%20Descriptions.md). ### Linear Transformations are simply maps between spaces See [Linear Transformations](Linear%20Transformations.md) for a deep dive. ### SVD Linear transformations, maps, can be decomposed! See [Singular Value Decomposition](Singular%20Value%20Decomposition.md) and [svd visualization.excalidraw](svd%20visualization.excalidraw.md) for more detail. ### How does linear regression fit in? See [Data and Linear Regression](Data%20and%20Linear%20Regression.md). --- Date: 20220714 Links to: Tags: #review References: * [Space, Transformations and Descriptions](Space,%20Transformations%20and%20Descriptions.md) * [Vectors and Representation](Vectors%20and%20Representation.md)