Noise Removal under Epipolar Constraints

As discussed in Section 10.3.1, triangulating noisy points produces non-intersecting lines whose intersection does not minimize the residual in image coordinates, for example under the Euclidean distance metric. We also saw that the best estimate of the noise-free points minimizes the quantity in equation 10.69 subject to the epipolar constraint 10.70. So far, however, given the Essential/Fundamental Matrix, this minimization has required the three-dimensional point as an auxiliary variable and an iterative optimization technique initialized, for example, using triangulation with the skew lines associated with the noisy points.

A global nonlinear technique makes it possible to obtain the optimal triangulation—the estimate of the image points—through a polynomial method (HS97) that requires finding the roots of a sixth-degree polynomial. As discussed more clearly in (Lin10), optimal triangulation can be viewed as the following minimization problem:

$$\min \left( \delta \mathbf{m}_2^{\top} \delta \mathbf{m}_2 + \delta \mathbf{m}_1^{\top} \delta \mathbf{m}_1 \right)$$ (10.87)

subject to the epipolar constraint

$$\hat{\mathbf{m}}_2^{\top} \mathbf{E} \hat{\mathbf{m}}_1 = (... ...f{E} (\mathbf{m}_1 - \mathbf{S}^\top \delta \mathbf{m}_1 ) = 0$$ (10.88)

where $\delta \mathbf{m}_1 = \mathbf{S} (\mathbf{m}_1 - \hat{\mathbf{m}}_1)$ and $\delta \mathbf{m}_2 = \mathbf{S} (\mathbf{m}_2 - \hat{\mathbf{m}}_2)$ have been defined, with

$$\mathbf{S} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix}$$ (10.89)

used to extract only the nonhomogeneous components from the vector. As seen previously, points $\left( \hat{\mathbf{m}}_1, \hat{\mathbf{m}}_2 \right)$ are the estimates of the noise-free points, whereas $\left( \mathbf{m}_1, \mathbf{m}_2 \right)$ are the observed points.

This constrained minimization problem can be solved using Lagrange multipliers:

$$\mathcal{L}(\delta \mathbf{m}_1, \delta \mathbf{m}_2, \lamb... ...athbf{E} (\mathbf{m}_1 - \mathbf{S}^\top \delta \mathbf{m}_1 )$$ (10.90)

The gradient of the Lagrangian vanishes at

$$\begin{array}{l} \hat{\mathbf{m}}_2^{\top} \mathbf{E} \hat{\... ...{E} \hat{ \mathbf{m} }_1 = \lambda \mathbf{n}_2 \\ \end{array}$$ (10.91)

yielding five constraints in five unknowns (the differential coordinates and $\lambda$). These constraints can be parameterized in terms of an auxiliary variable, from which the well-known sixth-degree equation is obtained. The approach is exactly the same whether image points and the Fundamental Matrix or camera points and the Essential Matrix are used.

Section (Lin10) also presents suboptimal iterative techniques with low computational cost, in which the epipolar constraint is nevertheless satisfied at every iteration.

Once the noise-free image points have been obtained, the three-dimensional point can be recovered using any triangulation technique, such as the skew-line intersection method discussed in Section 1.6.6 or the DLT method discussed in Section 10.3.1. An alternative formulation (KK95) states that, given two corresponding points expressed in camera coordinates $\hat{\mathbf{m}}'$ and $\hat{\mathbf{m}}$, the three-dimensional point formed by the intersection of the optical rays is

$$\mathbf{x} = \frac{ \left( \mathbf{t} \times \mathbf{R} \hat... ...bf{m}}' \right) \cdot \mathbf{z} }{ \Vert \mathbf{z} \Vert^2 }$$ (10.92)

where $\mathbf{z} = \hat{\mathbf{m}} \times \mathbf{R} \hat{\mathbf{m}}'$.