# Gradient The Gradient is the multi-variable generalization of the derivative. Specifically, if we have a function $f: \mathbb{R}^n \rightarrow \mathbb{R}$, it's gradient $\nabla f :\mathbb{R}^n \rightarrow \mathbb{R}^n$, is defined at point $p = (x_1, \dots, x_n)$ in $n$-dimensional space as the vector: $ \nabla f(p) = \begin{bmatrix} \frac{\partial f}{\partial x_1}(p) \\ \frac{\partial f}{\partial x_2}(p) \\ \vdots \\ \frac{\partial f}{\partial x_n}(p) \end{bmatrix} $ #### Key Intuitions * The gradient specifically is concerned with functions mapping from multiple dimensions back to $\mathbb{R}$. For function that map from multiple dimensions to multiple dimensions, such as $g: \mathbb{R}^n \rightarrow \mathbb{R}^n$, we use the [Jacobian](Jacobian-Matrix.md). * The gradient can be interpreted as the "direction and rate of fastest increase" * The reason that we often use the gradient in cost functions of machine learning is because if you *descend* along the direction of greatest descent (yielded to us via the gradient) you will be following the most rapid path to optimize (minimize in this case) your cost function. * The gradient is a vector valued function, meaning it takes a vector as input. This is in contrast to the derivative which takes a scalar as input. #### Comparison with derivative The biggest difference between the gradient and the derivative (see [here](https://www.youtube.com/watch?v=nJpONHO_X5o)), is that the gradient of a scalar field gives us a **vector field**, where the vectors point in the direction of steepest increase, with largest vectors indicating a greater change. On the other hand, $df$ gives us a **covector field**, also called a "differential" or "exterior derivative" or "1-form". %202-3%20screenshot.png) Above we can see the orientation of both fields is the same. Loosely speaking, we can note that the gradient is a vector while the derivative is a scalar. --- References: * [Gradient, khan, 3b1b](https://www.youtube.com/watch?v=tIpKfDc295M) * [Gradient and graph, khan, 3b1b](https://www.youtube.com/watch?v=_-02ze7tf08 )