Regression with Anisotropic Noise

When the observation noise is not isotropic, the Euclidean distance can no longer be used to measure the error; instead, it is necessary to use the Mahalanobis distance. Under this different metric, the cost function (4.49) is written as

$$S(\boldsymbol\beta) = \left( \mathbf{r}(\boldsymbol\beta)... ...\boldsymbol\Omega \left( \mathbf{r}(\boldsymbol\beta) \right)$$ (4.68)

where $\boldsymbol\Omega = \boldsymbol\Sigma^{-1}$ is the information matrix (information matrix), also called the concentration matrix or precision matrix. The Mahalanobis distance corresponds to the maximum-likelihood estimator in the case of zero-mean anisotropic Gaussian noise. In the special case where the covariance matrix is diagonal, this approach reduces to the weighted least-squares case.

The Taylor-series expansion of equation (4.68) is written as

$$\begin{array}{rl} S(\boldsymbol\beta + \boldsymbol\delta) ... ...\boldsymbol\delta^{\top}\mathbf{H}\boldsymbol\delta \end{array}$$ (4.69)

with $\mathbf{r}$ and $\mathbf{J}$ evaluated at point $\boldsymbol\beta$. The matrix

$$\mathbf{H} = \mathbf{J}^{\top}\boldsymbol\Omega\mathbf{J}$$ (4.70)

is the information matrix of the entire system, obtained by projecting the measurement error into parameter space through the Jacobian $\mathbf{J}$. For compactness, we also define

$$\mathbf{b} = \mathbf{J}^{\top}\boldsymbol\Omega\mathbf{r}.$$ (4.71)

The derivatives of the function $S$ therefore become

$$\frac{\partial S(\boldsymbol\beta+\boldsymbol\delta)} {\pa... ...elta} \approx 2\mathbf{b} + 2\mathbf{H}\boldsymbol\delta .$$ (4.72)

Setting the gradient to zero yields the linear system

$$\mathbf{H}\boldsymbol\delta = -\mathbf{b}$$ (4.73)

which represents the Gauss-Newton formulation in the case of anisotropic noise and constitutes the natural generalization of equation (4.52) obtained in the isotropic case.