Weighted Mean with Variance

Having multiple observations with different variances $\sigma^{2}_i$, the goal is to merge these various observations. This is the case, for example, when multiple measurements of the same observable are taken by different sensors at the same time, or with the same sensor for a quantity assumed to be constant but with variable observation noise over time. The objective is to obtain a weighted average of each individual observation of the form

$$\bar{x} = \sum_i w_i x_i$$ (2.60)

. The variance of the variable $\bar{x}$ will be defined as follows.

$$\sigma^2_{\bar{x}} = \sum_i w^{2}_i \sigma^2_{i}$$ (2.61)

. The optimal solution (maximum likelihood estimator) is obtained by minimizing this quantity under the additional constraint $\sum_i w_i = 1$.

The weight that minimizes this quantity is

$$w_i = \frac{ \frac{1}{\sigma^{2}_i} } { \sum_j \frac{1}{\sigma^{2}_j} }$$ (2.62)

In this way, the variance of the mean is lower than the variance of the individual measuring instruments and is given by

$$\sigma^{2}_{\bar{x}} = \frac{1}{\sum 1/\sigma^{2}_i}$$ (2.63)

A direct consequence is the ability to combine $n$ readings from the same sensor and the same observable (assuming the observation noise has constant variance) but at different time instances. The final variance is reduced to

$$\sigma^{2}_{\bar{x}} = \frac{ \sigma^{2}_0 } {n}$$ (2.64)

It is possible to construct this result iteratively through the sequence:

$$\bar{x}_{i+1} = (1 - k) \bar{x}_i + k x_{i+1} \quad k = \frac{\sigma^2_{\bar{x}}}{\sigma^2_{\bar{x}} + \sigma^2_{i+1}}$$ (2.65)

with $k$ blending factor. Written in this form, the estimate of the observable is in the same format as the one-dimensional Kalman filter (see this result in section 2.12.2): in the absence of process noise, the gain $k$ approaches zero.