# Radial Basis Function ### Infinite Dimensional Spaces The Gaussian RBF kernel is defined as: $K(x_i, x_j) = exp \big( -\gamma ||x_i - x_j ||^2 \big)$ where $\gamma$ is a parameter that dictates the width of the kernel, and $||x_i - x_j ||^2$ is the squared Euclidean distance between two points in the input space. **Expansion into Infinite-Dimensional Space** 1. Taylor Expansion of the Exponential Function: * The exponential function can be expanded using its Taylor series: $e^z = \sum_{k=0}^\infty \frac{z^k}{k!}$ * Applying this to the RBF kernel, let z = -\gamma \|x - x’\|^2 . The kernel then becomes: $K(x_i, x_j’) = \exp(-\gamma \|x_i - x_j\|^2) = \sum_{k=0}^\infty \frac{(-\gamma \|x_i - x_j\|^2)^k}{k!}$ * This series is infinite, hinting at the infinite-dimensional nature of the transformation. 2. Feature Space Interpretation: * Each term in this expansion can be interpreted as a dimension in the feature space. For example, $\|x_i - x_j\|^2$ can be expanded further into dot products and polynomial terms of the components of x and x’ , leading to an expansion in terms of monomials of increasing degree. * These monomials (e.g., $x_1^2, x_1x_2, x_2^2, \ldots$ ) represent dimensions in the feature space. As k increases, you get higher and higher degree polynomials, thus an increasingly larger (and theoretically infinite) number of dimensions. 3. Implicit Mapping to an Infinite-Dimensional Space: * The kernel function, through its expansion, essentially provides coefficients for these polynomial terms. Since the series is infinite, the number of polynomial terms (and thus dimensions) is also infinite. * What’s critical here is that no finite-dimensional vector can represent all these terms; hence, the feature space into which the data is mapped must be infinite-dimensional. --- Date: 20240524 Links to: Tags: References: * []()