Noise Removal under Epipolar Constraints

As seen in section 9.3.1, triangulating points affected by noise leads to non-concurrent lines whose intersection does not minimize the residual in image coordinates (for instance, under the metric of Euclidean distance). We have also observed that the best estimate of the noise-free points minimizes the quantity in equation 9.66 under the epipolar constraint 9.67. However, so far, given the knowledge of the Essential/Fundamental matrix, this minimization has required the three-dimensional point as an auxiliary variable and an (iterative) optimization technique initialized, for example, by leveraging the triangulation with skew lines of the noise-affected points.

There exists a global nonlinear technique that allows for optimal triangulation (the estimation of image points) through a polynomial method (HS97) that requires finding the roots of a 6th degree polynomial. As more clearly discussed in (Lin10), optimal triangulation can be viewed as the following minimization problem:

$$\min \left( \delta \mathbf{m}_2^{\top} \delta \mathbf{m}_2 + \delta \mathbf{m}_1^{\top} \delta \mathbf{m}_1 \right)$$ (9.77)

subject to the epipolar constraint

$$\hat{\mathbf{m}}_2^{\top} \mathbf{E} \hat{\mathbf{m}}_1 = (... ...f{E} (\mathbf{m}_1 - \mathbf{S}^\top \delta \mathbf{m}_1 ) = 0$$ (9.78)

having defined $\delta \mathbf{m}_1 = \mathbf{S} (\mathbf{m}_1 - \hat{\mathbf{m}}_1)$ and $\delta \mathbf{m}_2 = \mathbf{S} (\mathbf{m}_2 - \hat{\mathbf{m}}_2)$ where

$$\mathbf{S} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix}$$ (9.79)

is used to extract only the non-homogeneous components from the vector. As previously seen, the points $\left( \hat{\mathbf{m}}_1, \hat{\mathbf{m}}_2 \right)$ are the estimates of the noise-free points while $\left( \mathbf{m}_1, \mathbf{m}_2 \right)$ are the observed points.

This constrained minimization problem can be solved using the method of Lagrange multipliers:

$$\mathcal{L}(\delta \mathbf{m}_1, \delta \mathbf{m}_2, \lamb... ...athbf{E} (\mathbf{m}_1 - \mathbf{S}^\top \delta \mathbf{m}_1 )$$ (9.80)

The gradient of the Lagrangian is set to zero at

$$\begin{array}{l} \hat{\mathbf{m}}_2^{\top} \mathbf{E} \hat{\... ...{E} \hat{ \mathbf{m} }_1 = \lambda \mathbf{n}_2 \\ \end{array}$$ (9.81)

from which we obtain 5 constraints in 5 unknowns (the differential coordinates and $\lambda$). These constraints can be parameterized as a function of an auxiliary variable, leading to the famous sixth-degree equation. This approach is exactly the same whether using image points with the Fundamental matrix or camera points with the Essential matrix.

In (Lin10), sub-optimal, iterative techniques with low computational cost are also proposed, where the epipolar constraint is still satisfied at each iteration.

Once the image points unaffected by noise are obtained, it is possible to derive the three-dimensional point using any triangulation technique (skew lines in section 1.5.6 or the DLT in section 9.3.1). An alternative formulation (KK95), given two homologous points expressed in camera coordinates $\hat{\mathbf{m}}'$ and $\hat{\mathbf{m}}$, defines the three-dimensional point formed by the intersection of the optical rays as

$$\mathbf{x} = \frac{ \left( \mathbf{t} \times \mathbf{R} \hat... ...bf{m}}' \right) \cdot \mathbf{z} }{ \Vert \mathbf{z} \Vert^2 }$$ (9.82)

where $\mathbf{z} = \hat{\mathbf{m}} \times \mathbf{R} \hat{\mathbf{m}}'$.