# Laws of Large Numbers The laws of large numbers are a collection of results in probability theory that describe the behavior of the arithmetic average of $n$ random variables for large $n$. For any $n$ random variables, $X_1, \dots, X_n$, the *arithmetic average* is the rv: $\frac{1}{n} \sum_{i=1}^n X_i$ This is known as the **sample average**. If $X_1, \dots, X_n$ are viewed as successive variables in time, this sample average is called the **time average**. > Under fairly general assumptions, the standard deviation of the sample average goes to $0$ with increasing $n$, and, in various ways depending on the assumptions, the sample average approaches the mean. This is central to the study of [Stochastic Processes](Stochastic%20Processes.md) because they allow us to relate *time averages* (i.e. the average over time of individual sample paths) to **ensemble averages** (i.e., the mean of the value of the process at a given time). ### Weak Law of Large Numbers (WLLN) For each integer $n \geq 1$, let $S_n = X_1 + \dots + X_n$ be the sum of $n$ IID rvs with a finite variance. Then: $\lim_{n \rightarrow \infty} P \Big\{ \Big | \frac{S_n}{n} \Big| - \bar{X} > \epsilon \Big\} = 0$ The above is saying that $\frac{S_n}{n}$ **converges in probability** to $\bar{X}$.    --- Date: 20220803 Links to: Tags: #review References: * [Nathaniel Dake Blog](https://www.nathanieldake.com/Mathematics/04-Statistics-01-Introduction.html) *