Noise removal under epipolar constraints

As seen in section 10.3.1, triangulating noisy points leads to non-intersecting lines whose intersection does not minimize the residual in image coordinates (for example, under the Euclidean-distance metric). We also saw that the best estimate of the noise-free points minimizes the quantity in equation 10.72 subject to the epipolar constraint 10.73. So far, however, given the Essential/Fundamental Matrix, this minimization has required the three-dimensional point as an auxiliary variable and an (iterative) optimization technique initialized, for example, by exploiting the triangulation of noisy points using skew lines.

A global nonlinear technique makes it possible to obtain the optimal triangulation (the estimate of the image points) through a polynomial method (HS97) that requires finding the roots of a sixth-degree polynomial. As discussed more clearly in (Lin10), optimal triangulation can be viewed as the following minimization problem:

$$\min \left( \delta \mathbf{m}_2^{\top} \delta \mathbf{m}_2 + \delta \mathbf{m}_1^{\top} \delta \mathbf{m}_1 \right)$$ (10.90)

subject to the epipolar constraint

$$\hat{\mathbf{m}}_2^{\top} \mathbf{E} \hat{\mathbf{m}}_1 = (... ...f{E} (\mathbf{m}_1 - \mathbf{S}^\top \delta \mathbf{m}_1 ) = 0$$ (10.91)

where $\delta \mathbf{m}_1 = \mathbf{S} (\mathbf{m}_1 - \hat{\mathbf{m}}_1)$ and $\delta \mathbf{m}_2 = \mathbf{S} (\mathbf{m}_2 - \hat{\mathbf{m}}_2)$ have been defined, and the matrix

$$\mathbf{S} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix}$$ (10.92)

is used to extract only the nonhomogeneous components from the vector. As seen previously, the points $\left( \hat{\mathbf{m}}_1, \hat{\mathbf{m}}_2 \right)$ are the estimates of the noise-free points, whereas $\left( \mathbf{m}_1, \mathbf{m}_2 \right)$ are the observed points.

This constrained minimization problem can be solved using Lagrange multipliers:

$$\mathcal{L}(\delta \mathbf{m}_1, \delta \mathbf{m}_2, \lamb... ...athbf{E} (\mathbf{m}_1 - \mathbf{S}^\top \delta \mathbf{m}_1 )$$ (10.93)

The gradient of the Lagrangian vanishes at

$$\begin{array}{l} \hat{\mathbf{m}}_2^{\top} \mathbf{E} \hat{\... ...{E} \hat{ \mathbf{m} }_1 = \lambda \mathbf{n}_2 \\ \end{array}$$ (10.94)

yielding five constraints in five unknowns (the differential coordinates and $\lambda$). These constraints can be parameterized as a function of an auxiliary variable, from which the well-known sixth-degree equation is obtained. This approach is exactly the same whether image points and the Fundamental Matrix or camera points and the Essential Matrix are used.

Also in (Lin10), suboptimal iterative techniques with low computational cost are presented, in which the epipolar constraint is nevertheless satisfied at every iteration.

Once the noise-free image points have been obtained, the three-dimensional point can be recovered using any triangulation technique (the skew-line method of section 1.6.6 or the DLT of section 10.3.1). An alternative formulation (KK95), given two corresponding points expressed in camera coordinates $\hat{\mathbf{m}}'$ and $\hat{\mathbf{m}}$, is that the three-dimensional point formed by the intersection of the optical rays is

$$\mathbf{x} = \frac{ \left( \mathbf{t} \times \mathbf{R} \hat... ...bf{m}}' \right) \cdot \mathbf{z} }{ \Vert \mathbf{z} \Vert^2 }$$ (10.95)

where $\mathbf{z} = \hat{\mathbf{m}} \times \mathbf{R} \hat{\mathbf{m}}'$.