Factorization of the Essential Matrix

The preceding sections showed how, using at least five correspondences between corresponding points, it is possible to estimate the Essential Matrix $\mathbf{E}$ that encodes the relative pose between two calibrated cameras.

The goal is now to invert this relationship and extract from the Essential Matrix the geometric parameters that generated it, namely the relative rotation $\mathbf{R}$ and the direction of the translation $\mathbf{t}$.

From the definition

$$\mathbf{E} = [\mathbf{t}]_{\times}\mathbf{R}$$ (10.76)

it follows immediately that

$$\mathbf{E}\mathbf{E}^{\top} = [\mathbf{t}]_{\times} [\mathbf{t}]_{\times}^{\top}$$ (10.77)

and therefore the symmetric matrix $\mathbf{E}\mathbf{E}^{\top}$ depends only on the translation vector and not on the relative rotation between the two cameras. Consequently, the Essential Matrix contains both rotation and translation information in coupled form.

In practice, factorization is performed directly through Singular Value Decomposition (SVD). Let

$$\mathbf{E} = \mathbf{U} \mathbf{D} \mathbf{V}^{\top}$$ (10.78)

with

$$\mathbf{D} = \diag (1,1,0)$$ (10.79)

in the case of an ideally normalized Essential Matrix. If the estimated matrix does not exactly satisfy this constraint, it can be projected onto the space of Essential Matrices as discussed in section 10.4.1.

Also define the matrix

$$\mathbf{W} = \begin{bmatrix} 0 & -1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix}.$$ (10.80)

The SVD yields two possible rotation matrices^10.5:

$$\mathbf{R}_1 = \mathbf{U} \mathbf{W} \mathbf{V}^{\top}$$ (10.81)

$$\mathbf{R}_2 = \mathbf{U} \mathbf{W}^{\top} \mathbf{V}^{\top}.$$ (10.82)

The direction of the translation is instead given by the third column of matrix $\mathbf{U}$:

$$\mathbf{t} = \pm \mathbf{u}_3$$ (10.83)

where $\mathbf{u}_3$ denotes the last column of $\mathbf{U}$.

The sign ambiguity arises because the Essential Matrix determines the translation only up to a multiplicative factor. In particular, both $\mathbf{t}$ and $-\mathbf{t}$ generate the same epipolar geometry.

Thus, four possible factorizations of the Essential Matrix are obtained:

$$(\mathbf{R}_1,+\mathbf{t}) \qquad (\mathbf{R}_1,-\mathbf{t})$$ (10.84)

$$(\mathbf{R}_2,+\mathbf{t}) \qquad (\mathbf{R}_2,-\mathbf{t}).$$ (10.85)

All these solutions are compatible with the same epipolar geometry. The corresponding Essential Matrices differ at most by a scale factor and are therefore indistinguishable using only the epipolar constraint. Identifying the physically correct configuration requires introducing an additional constraint known as the chirality constraint.

Footnotes

... matrices^10.5

To obtain proper rotation matrices, it is necessary to verify that $\det(\mathbf{U}\mathbf{V}^{\top}) = 1$. If the determinant is negative, the sign of the last column of $\mathbf{U}$ (or, equivalently, of $\mathbf{V}$) can be changed before reconstructing the Essential Matrix.