Calibration using the Sturm–Maybank–Zhang method

Zhang (Zha99) and, independently, Sturm and Maybank (SM99) identified a method for obtaining a linear equation to recover the camera parameters, while also updating the calibration techniques developed mainly by Tsai (Tsa87) and others (WM94), which were still valid but by then dated to the 1980s.

This technique uses several homography matrices $\mathbf{H}$ obtained by observing a plane (for example, a calibration grid with evenly spaced markers) and attempts to recover the camera's intrinsic parameters explicitly from them. As discussed previously, matrix $\mathbf{H}$, the homographic transformation of a plane, has 8 degrees of freedom, but it is not directly possible to recover the 10 explicit parameters that generated it. Methods for obtaining the homography matrix from correspondences between image points and plane points are discussed in Section 9.5.1.

Matrix $\mathbf{H}$, and in particular Equation (9.36), can be written explicitly as

$$\mathbf{H} = \begin{bmatrix} \mathbf{h}_1 & \mathbf {h}_2 ... ...atrix} \mathbf{r}_1 & \mathbf {r}_2 & \mathbf{t} \end{bmatrix}$$ (9.72)

where $\lambda$ indicates the presence of an unknown multiplicative factor in the computation of the homography matrix. We focus on the part of the rotation matrix formed by the column vectors $\mathbf{r}_1$ and $\mathbf{r}_2$, which are orthonormal to each other.

Despite the presence of factor $\lambda$, it is possible to express relations based on the orthogonality of vectors $\mathbf{r}_1$ and $\mathbf{r}_2$ in order to impose the following two constraints:

$$\begin{array}{l} \mathbf{h}_1^{\top} \mathbf{W} \mathbf{h}... ...h}_1 = \mathbf{h}_2^{\top} \mathbf{W} \mathbf{h}_2 \end{array}$$ (9.73)

where $\mathbf{W}$ is defined, neglecting skew for simplicity, as

$$\mathbf{W} = (\mathbf{K}^{-1})^{\top} \mathbf{K}^{-1} = \beg... ...dfrac{u_0^2}{k_u^2} + \dfrac{v_0^2}{k_v^2} + 1\\ \end{bmatrix}$$ (9.74)

a symmetric matrix. This equation is the equation of a conic and is, in fact, the equation of the “absolute conic” (LF97).

The 4 (or 5 unknowns when skew is not neglected) in matrix $\mathbf{W}$ under the 2 constraints (9.73) can be solved using at least 2 (or 3) different planes; that is, matrices $\mathbf{H}$ whose columns are not linearly dependent on one another.

Once matrix $\mathbf{W}$ has been obtained, the original matrix can finally be determined using Cholesky decomposition. Alternatively, Zhang provides equations for obtaining the camera's intrinsic parameters directly from matrix $\mathbf{W}$. In fact, $\mathbf{h}_i^{\top} \mathbf{W} \mathbf{h}_j = \mathbf{v}_{ij}^{\top} \mathbf{w}$ can be transformed using suitable values of vector $\mathbf{v}_{ij}$ and with $\mathbf{w}$, the vector to be determined, containing the nonzero values of the upper-triangular matrix $\mathbf{W}$. In this way, the system of equations (9.73) is transformed into the solution of a homogeneous linear system in $\mathbf{w}$.

Once the intrinsic parameters and matrix $\mathbf{K}$ have been determined, the rotation and translation can be estimated for each homography matrix $\mathbf{H}$ used during the optimization stage:

$$\begin{bmatrix}\mathbf{r}_1 & \mathbf{r}_2 & \mathbf{t} \end{bmatrix} = \lambda \mathbf{K}^{-1}\mathbf{H}$$ (9.75)

Columns $\mathbf{r}_1$ and $\mathbf{r}_2$ are normally sufficient to recover the rotation angles. All extrinsic parameters can be recovered from each grid, making it possible to measure the reprojection error.

The system as a whole is nevertheless ill-conditioned, and obtaining a stable solution after repeated trials is difficult. The values obtained through this linear technique are useful, however, as the starting point for a Maximum Likelihood Estimation stage that minimizes the reprojection errors (Section 9.5.6).

One final note: in his paper, Zhang assumes that the Principal Point coincides with the distortion center, which is generally not exact.