Maximum Likelihood Estimation

When SVD is used to enforce the constraints, the resulting Fundamental (or Essential) Matrix fully satisfies the requirements for being Fundamental (or Essential), but it is only the matrix closest, according to a particular norm—in this case, the Frobenius norm—to the one obtained from the linear system. This solution is therefore not optimal either, since it still does not account for how the error propagates from the input points through the transformation: it is still an algebraic rather than a geometric solution.

A first technique that minimizes the geometric error consists in exploiting the distance between points and the epipolar lines generated by the Fundamental Matrix (epipolar distance). Even intuitively, the distance between a point $\mathbf {p}_2$ and the epipolar line $\mathbf{F}\mathbf{p}_1$ can be used as a metric for estimating the geometric error:

$$d \left( \mathbf{p}_{2}, \mathbf{F} \mathbf{p}_{1} \right) ... ...right)_1^{2} + \left( \mathbf{F}\mathbf{p}_1 \right)_2^{2} } }$$ (10.65)

where $(.)_i$ denotes the $i$-th component of the vector (see Section 1.6.3 for the point-to-line distance equation). The smaller the distance, the more closely matrix $\mathbf{F}$ represents the matrix relating corresponding points.

Since this error can be computed for both the first and the second image, both contributions should be minimized together. This metric can be used to define a cost function that minimizes the error symmetrically (symmetric transfer error) between the two images:

$$\min_\mathbf{F} \sum_i \left( d \left( \mathbf{p}_{1,i} , \... ...}_{2,i} , \mathbf{F}^{\top} \mathbf{p}_{1,i} \right)^2 \right)$$ (10.66)

Here too, an 8-unknown solution can be sought, but obtaining a robust solution requires constraining $\mathbf{F}$ to have rank 2.

As an alternative to the Symmetric Transfer Error, the first-order approximation of the distance between the points and the function (Sampson error, Section 4.3.8) is often used in the literature. An approximate distance between the corresponding image points $\left( \mathbf{p}_{1}, \mathbf{p}_{2} \right)$ and the manifold $\hat{\mathbf{p}_{2}}^{\top}\mathbf{F} \hat{\mathbf{p}_{1} }=0$ can thus be defined through the metric

$$r\left( \mathbf{p}_{1}, \mathbf{p}_{2}, \mathbf{F} \right) ... ... \mathbf{p}_2)_1^2 + (\mathbf{F}^{\top} \mathbf{p}_2)_2^2 } }$$ (10.67)

where $(.)_i$ again denotes the $i$-th component of the vector. Using this approximate metric, while maintaining the additional constraint $\det \mathbf{F} = 0$, it is possible to minimize

$$\min_{\mathbf{F}} \sum_{i=1}^{n} r\left( \mathbf{p}_{1,i}, \mathbf{p}_{2,i}, \mathbf{F} \right)^{2}$$ (10.68)

Neither the Symmetric Transfer Error nor the Sampson distance, although better metrics than the algebraic estimate, is the optimal estimator. The Maximum Likelihood Estimation for the Fundamental Matrix would in fact be obtained using a cost function of the form

$$\min_\mathbf{F} \sum_i \Vert \mathbf{p}_{1,i} - \hat{\mathb... ...rt^2 + \Vert \mathbf{p}_{2,i} - \hat{\mathbf{p}}_{2,i} \Vert^2$$ (10.69)

where $\hat{\mathbf{p}}_{1,i}$ and $\hat{\mathbf{p}}_{2,i}$ denote the exact points, and $\mathbf{p}_{1,i}$ and $\mathbf{p}_{2,i}$ the corresponding measured points affected by zero-mean white Gaussian noise. Cost function (10.69) must be minimized subject to the constraint

$$\hat{\mathbf{p}}^{\top}_{2,i} \mathbf{ F } \hat{\mathbf{p}}_{1,i} = 0$$ (10.70)

and to the additional constraints arising from the nature of $\mathbf{F}$. In this case, the exact points $\hat{\mathbf{p}}_{1,i}$ and $\hat{\mathbf{p}}_{2,i}$ become part of the problem as auxiliary variables (subsidiary variables). However, introducing points $\hat{\mathbf{p}}_{1,i}$ and $\hat{\mathbf{p}}_{2,i}$ as unknowns makes the problem unsolvable, since there would always be more unknowns than constraints.

To solve this problem, the problem of computing the Essential or Fundamental Matrix must be combined with that of three-dimensional reconstruction, with the three-dimensional coordinates of the observed point $\hat{\mathbf{x}}_{i}$ used directly as the auxiliary variable. The Essential Matrix can be obtained when the intrinsic parameters of the two sensors are known. In this case, the nonlinear system that projects the auxiliary variable $\hat{\mathbf{x}}_{i}$ onto the respective observations in the two sensors can be used:

$$\begin{array}{l} \hat{\mathbf{p}}_{1,i} \equiv \mathbf{K}_... ...f{R} \hat{\mathbf{x}}_{i} + \mathbf{t} \right) \\ \end{array}$$ (10.71)

where matrix $\mathbf{R}$ can be expressed using a 3-parameter parameterization (see Section A), whereas vector $\mathbf{t}$ must be represented using a 2-parameter parameterization, since its scale remains an unknown factor. By inserting constraints (10.71) into equation (10.69), the objective of recovering the Essential Matrix is transformed into that of directly recovering the relative parameters between the two sensors. If required, once the relative pose between the sensors has been obtained, the Essential Matrix can be obtained by directly applying definition (10.43).

When the intrinsic parameters are unavailable, as in the estimation of the Fundamental Matrix, a true three-dimensional reconstruction of the scene cannot be performed precisely because these parameters are missing. It is nevertheless possible to use fictitious perspective projections by setting $\mathbf{K}_1=\mathbf{I}$ and obtaining constraints of the form:

$$\begin{array}{l} \hat{\mathbf{p}}_{1,i} \equiv \hat{\mathb... ...}_{2,i} \equiv \mathbf{P} \hat{\mathbf{x}}_{i} \\ \end{array}$$ (10.72)

using directly the auxiliary variable $\hat{\mathbf{x}}_{i}$, a coordinate that is therefore known up to an affine transformation $\mathbf{K}_1^{-1}$, namely, the intrinsic parameters of camera 1.

By inserting constraints (10.72) into equation (10.69), the objective of recovering the Fundamental Matrix is likewise transformed into that of recovering the parameters of the projective matrix $\mathbf{P}$. Using camera matrix $\mathbf{P}=\left[ \mathbf{R}' \vert \mathbf{t}' \right]$, a fictitious camera matrix, it is finally possible to obtain $\mathbf{F}$ by directly applying definition (10.43), although matrix $\mathbf{R}'$ is not a rotation matrix.

The maximum-likelihood estimate of the Fundamental Matrix, which is probabilistically correct, nevertheless requires substantial computational resources: in addition to the 12 global unknowns needed to estimate $\mathbf{P}$ (as opposed to the 5 for the Essential Matrix), 3 additional unknowns are introduced into the problem for each point pair being minimized.

Finally, as a concluding warning, techniques such as RANSAC (Section 4.12) are widely used for the optimal estimation of the matrices in the presence of possible outliers in the scene.