Rectification

The case of pure rotation is a special scenario: a camera rotating around its optical center captures images of a 3D scene as if the scene were projected onto a plane infinitely far from the pin-hole.

In the case where $\mathbf{t} = 0$, that is, the coordinates of the two pinholes of the two views are coincident $\mathbf{t}_1 = \mathbf{t}_2$, the transformation (8.31) reduces in dimension and results in an equation compatible with a homography, and consequently valid for any point in the image, regardless of the presence of a dominant plane. Therefore, when the pinhole is common between the two views (thus representing a pure rotation or modification of intrinsic parameters), it is possible to achieve a perfect transformation for all points in the image. This process of projecting points from one camera to another by modifying intrinsic parameters and rotation is referred to as rectification.

To rectify an image, that is, to generate a dense image 1 from the points of image 2, it is necessary to use the homography matrix

$$\mathbf{H}_{1,2}=\mathbf{K}_2 \mathbf{R}_2\mathbf{R}_1^{-1} \mathbf{K}^{-1}_1$$ (8.37)

which allows us to derive all the points of image 1 from the points of image 2. Specifically, for each pixel $(u_1, v_1)$ of the image to be generated, the homographic transformation $\mathbf{H}$ is applied, and the point $(u_2,v_2)$ of the source image is obtained from which to copy the pixel value.

Through the transformation (8.37), it is possible to convert an image captured by a camera with parameters $(\mathbf{K}_2,\mathbf{R}_2)$ into an image of a virtual camera with parameters $(\mathbf{K}_1, \mathbf{R}_1)$.

The discussion applies to all homographies, a method for obtaining the matrix $\mathbf{H}$ without knowledge of the intrinsic and extrinsic parameters but solely through correspondences between the views of the two cameras is presented in section 8.5.1. The homography can then be factored to retrieve the parameters that generated it.