Mixture Models

Mixture models are a type of density model consisting of a certain number of density functions, usually Gaussians (Gaussian Mixture Models), which are combined to provide a multimodal density. Mixture models make it possible to represent probability distributions in the presence of subpopulations. For example, they can be used to model the colors of an object and exploit this information for tracking or color-based segmentation.

A mixture model is a mathematical formalism capable of modeling a probability distribution as the sum of parametric distributions. Mathematically,

$$p_{X}(x) = \sum_{k = 1}^{n} a_{k} h(x \vert \lambda_k)$$ (2.21)

where $p_{X}(x)$ is the modeled distribution function, $n$ is the number of components in the model, and $a_k$ is the proportionality factor of component $k$. By definition, $0 < a_k < 1$, $\forall k = 1,\dots,n$, and $a_{1} + \cdots + a_{n} = 1$. $h(x \vert \lambda_k)$ is a probability distribution parameterized by a vector (in general) $\lambda_k$. In Gaussian mixture models, the parameter vector consists of the mean and variance of the individual components.

Mixture models are often used when $h(x)$ is known, $p_{X}(x)$ can be sampled, and one only wants to determine the parameters $a_k$ and $\lambda_k$. A practical example in which this formalism is used is the analysis of a population made up of distinct subpopulations.