The Gaussian Distribution

Figure 2.1: Gaussian distribution

The Gaussian distribution is one of the most widely used probability distributions in practical problems, since it models a large portion of the probability distributions encountered in real-world events. In this document, it is used in filters (Section 3), Bayesian classifiers (Section 5.2), and LDA (Section 5.3).

Definizione 6   The standard Gaussian distribution, denoted by $\mathcal{N}(0;1)$, is the one with density

$$p(x) = \frac{1}{\sqrt{2\pi}} e ^ { \left( - \dfrac{1}{2}x^{2} \right) }$$ (2.15)

Definizione 7   The general Gaussian distribution $\mathcal{N}(\mu;\sigma^{2})$, with $\mu\in\mathbb{R}$ and $\sigma>0$, is obtained from the standard distribution through the transformation $x \mapsto \sigma x + \mu$.

In the univariate case (univariate Gaussian), the Gaussian has the following distribution function:

$$p(x) = \frac{1}{\sigma \sqrt{2 \pi} } e^{ -\dfrac{1}{2} \left( \dfrac{x - \mu}{ \sigma } \right)^2}$$ (2.16)

where $\mu$ is the expected value and $\sigma^2$ is the variance. Within $\pm \sigma$ of $\mu$ lies 68% of the probability, within $\pm 2 \sigma$ lies 95%, and within $\pm 3 \sigma$ lies 99.7%.

The multivariate Gaussian distribution (multidimensional Gaussian) is defined by a vector $\boldsymbol\mu$ of dimension $n$, representing the expected values of the various components, and by a covariance matrix $\boldsymbol\Sigma$ of dimensions $n \times n$:

$$p(\mathbf{x}) = \frac{1}{ (2 \pi)^{\frac{n}{2}} \sqrt{\vert\... ...)^{\top} \boldsymbol\Sigma^{-1} (\mathbf{x}-\boldsymbol\mu) }$$ (2.17)

a normal distribution with mean $\boldsymbol\mu = \left[ \mu_1, \mu_2, \dots \mu_n \right]^{T}$ and covariance $\boldsymbol\Sigma = \begin{bmatrix} \sigma_{11} & \cdots & \sigma_{1n} \\ \vdots & \ddots & \vdots \\ \sigma_{n1} & \cdots & \sigma_{nn} \\ \end{bmatrix}$.

It can be anticipated that the quantity in the exponent of equation (2.17) is the Mahalanobis distance (Section 2.4) between $\mathbf {x}$ and $\boldsymbol\mu$.

When the random variables are mutually independent and have equal variance, matrix $\boldsymbol\Sigma$ is a diagonal matrix whose entries are all equal to $\sigma^{2}$, and the multivariate normal probability distribution reduces to

$$p(\mathbf{x}) = \frac{1}{ (2 \pi \sigma^{2} )^{n/2} } e^{ - ... ...ac{ \vert \mathbf{x} - \boldsymbol\mu\vert^2}{2 \sigma^{2}} }$$ (2.18)