Reconstruction with Rectified Cameras

If the (corresponding) points in the two images of a stereo pair lay on the same image row (that is, had the same $v$ coordinate), highly optimized code could be used to search for correspondences (LZ99) and obtain dense disparity images.

There is a particular configuration of two cameras in which this condition is satisfied, namely, when the intrinsic parameters are identical and the optical axes are oriented perpendicularly to the vector connecting the pin-holes. For example, if the vector connecting the pin-holes lies along the $y$ axis, the stereo-pair configuration that produces corresponding points on the same row is the one with rotation angles $\rho=0$ and $\gamma = 0$ and equal pitch angles.

The software procedure for obtaining this configuration when the constraint is not satisfied in hardware consists of rectification (see 9.3.4): starting from an image acquired with a given set of (hardware) parameters, a new view of the same scene is generated with the desired intrinsic parameters and yaw, pitch, and roll.

Based on this observation, the three-dimensional reconstruction problem can always be reduced to a pair of cameras perfectly aligned with each other and with the axes, together with a roto-translation for transforming world coordinates from this sensor system to the actual system.

The following sections present the particular cases of cameras perfectly aligned with the axes, cameras aligned but tilted (with a nonzero pitch angle), and arbitrarily oriented cameras.