Reconstruction with Rectified Cameras

If corresponding points in the two images of a stereo pair lay on the same image row (that is, if they had the same coordinate $v$), highly optimized code could be used to search for correspondences (LZ99) and obtain dense disparity images.

There is a particular configuration of two cameras in which this condition is satisfied: their intrinsic parameters are equal and their optical axes are oriented perpendicular to the vector joining the pin-holes. For example, when the vector joining the pin-holes lies along the $y$ axis, the stereo-pair configuration that makes it possible to obtain corresponding points on the same row is the one with rotation angles $\rho=0$ and $\gamma = 0$ and equal pitch angles.

The software procedure for obtaining this configuration when the constraint is not satisfied in hardware consists of rectification (see 9.3.4). Starting from an image acquired with a given set of (hardware) parameters, a new view of the same scene is generated with the desired intrinsic parameters, yaw, pitch, and roll.

This observation means that the three-dimensional reconstruction problem can always be reduced to a pair of perfectly aligned cameras, both with respect to each other and to the axes, together with a rigid transformation for converting world coordinates from this sensor system into the actual physical system.

The following sections will address the special cases of cameras perfectly aligned with the axes, aligned cameras with a nonzero pitch angle, and arbitrarily oriented cameras.