Mixture Models

Mixture models are a type of density model consisting of a certain number of density functions, usually Gaussian functions (Gaussian Mixture Models), which are combined to provide a multimodal density. Mixture models make it possible to represent probability distributions in the presence of subpopulations. For example, they can be used to model the colors of an object and exploit this information for tracking or color-based segmentation.

A mixture model is a mathematical formalism sufficient for modeling a probability distribution as the sum of parametric distributions. Mathematically,

$$p_{X}(x) = \sum_{k = 1}^{n} a_{k} h(x \vert \lambda_k)$$ (2.21)

where $p_{X}(x)$ is the modeled distribution function, $n$ is the number of components in the model, and $a_k$ is the proportion factor of component $k$. By definition, $0 < a_k < 1$ $\forall k = 1,\dots,n$ and $a_{1} + \cdots + a_{n} = 1$. $h(x \vert \lambda_k)$ is a probability distribution parameterized by a vector (in general) $\lambda_k$. In the case of Gaussian mixture models, the parameter vector consists of the mean and variance of the individual components.

Mixture models are often used when $h(x)$ is known, $p_{X}(x)$ can be sampled, and one only wants to determine the parameters $a_k$ and $\lambda_k$. An example of a practical situation in which this formalism is used is the analysis of a population composed of distinct subpopulations.