Mixture Models

Mixture models are a type of density model consisting of a certain number of density functions, typically Gaussian (Gaussian Mixture Models), which are combined to provide a multimodal density. Mixture models allow for the representation of probability distributions in the presence of subpopulations. They can, for example, be used to model the colors of an object and leverage this information for tracking or color-based segmentation.

The mixture model is a mathematical formalism sufficient to model a probability distribution as a sum of parametric distributions. In mathematical terms,

$$p_{X}(x) = \sum_{k = 1}^{n} a_{k} h(x \vert \lambda_k)$$ (2.19)

where $p_{X}(x)$ is the modeled distribution function, $n$ is the number of components in the model, and $a_k$ is the proportion factor of component $k$. By definition, $0 < a_k < 1$, $\forall k = 1,\dots,n$, and $a_{1} + \cdots + a_{n} = 1$. $h(x \vert \lambda_k)$ is a probability distribution parameterized by a (generally) $\lambda_k$ vector. In the case of Gaussian mixture models, the parameter vector consists of the mean and variance of the individual components.

Mixture models are often used when $h(x)$ is known, $p_{X}(x)$ can be sampled, and the goal is to determine only the parameters $a_k$ and $\lambda_k$. An example of a practical situation where this formalism is employed is when analyzing a population composed of distinct subpopulations.