Factorization of the Essential Matrix

The preceding sections showed that, using at least five correspondences between homologous points, it is possible to estimate the Essential Matrix $\mathbf{E}$, which encodes the relative pose between two calibrated cameras.

The objective is now to invert this relationship and extract from the Essential Matrix the geometric parameters that generated it, namely, the relative rotation $\mathbf{R}$ and the direction of the translation $\mathbf{t}$.

From the definition

$$\mathbf{E} = [\mathbf{t}]_{\times}\mathbf{R}$$ (10.73)

it immediately follows that

$$\mathbf{E}\mathbf{E}^{\top} = [\mathbf{t}]_{\times} [\mathbf{t}]_{\times}^{\top}$$ (10.74)

and therefore the symmetric matrix $\mathbf{E}\mathbf{E}^{\top}$ depends solely on the translation vector and not on the relative rotation between the two cameras. Consequently, the Essential Matrix contains both rotation and translation information in coupled form.

In practice, factorization is performed directly through Singular Value Decomposition (SVD). Let

$$\mathbf{E} = \mathbf{U} \mathbf{D} \mathbf{V}^{\top}$$ (10.75)

with

$$\mathbf{D} = \diag (1,1,0)$$ (10.76)

in the case of an ideally normalized Essential Matrix. If the estimated matrix does not exactly satisfy this constraint, it can be projected into the space of Essential Matrices as discussed in Section 10.4.1.

Also define the matrix

$$\mathbf{W} = \begin{bmatrix} 0 & -1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix}.$$ (10.77)

The SVD yields two possible rotation matrices:

$$\mathbf{R}_1 = \mathbf{U} \mathbf{W} \mathbf{V}^{\top}$$ (10.78)

$$\mathbf{R}_2 = \mathbf{U} \mathbf{W}^{\top} \mathbf{V}^{\top}.$$ (10.79)

The translation direction is instead given by the third column of matrix $\mathbf{U}$:

$$\mathbf{t} = \pm \mathbf{u}_3$$ (10.80)

where $\mathbf{u}_3$ denotes the last column of $\mathbf{U}$.

The sign ambiguity arises because the Essential Matrix determines the translation only up to a multiplicative factor. In particular, both $\mathbf{t}$ and $-\mathbf{t}$ generate the same epipolar geometry.

Thus, four possible factorizations of the Essential Matrix are obtained:

$$(\mathbf{R}_1,+\mathbf{t}) \qquad (\mathbf{R}_1,-\mathbf{t})$$ (10.81)

$$(\mathbf{R}_2,+\mathbf{t}) \qquad (\mathbf{R}_2,-\mathbf{t}).$$ (10.82)

All these solutions produce the same Essential Matrix and are therefore indistinguishable using only the epipolar constraint. Identifying the physically correct configuration requires an additional constraint, known as the chirality constraint.