Bundle Adjustment

Visual odometry is a dead reckoning algorithm and is therefore subject to drift. Indeed, composing many relative-motion estimates can progressively accumulate estimation errors.

Extending the minimization to a complete sequence of frames yields a problem of the form

$$\min_{\{\mathbf{R}_i,\mathbf{t}_i,\mathbf{x}_j\}} \sum_{i,j} \left\Vert \mathbf{p}_{ij}-\hat{\mathbf{p}}_{ij} \right\Vert^2,$$ (10.102)

where the camera poses and the positions of the observed three-dimensional points are optimized simultaneously.

When the camera poses and the positions of the three-dimensional points observed throughout a sequence are estimated simultaneously by minimizing the reprojection errors, the process is called Bundle Adjustment.

The concept of Bundle Adjustment, introduced in photogrammetry and subsequently adopted in Computer Vision (see the survey (TMHF00)), therefore denotes a multivariable minimization aimed at jointly obtaining a three-dimensional reconstruction, the camera poses in an image sequence, and, where applicable, the cameras' intrinsic parameters.

This is a nonlinear optimization technique that uses the reprojection errors of the observed points as its cost function, in the same form as equation (10.101). Since the same feature may be observed in several images, a single observation contributes to the estimation of multiple poses and three-dimensional points. The problem therefore cannot be decomposed into a sequence of independent visual-odometry problems: the different images and shared points must be considered jointly.

The number of variables can become very large, and the problem is generally nonconvex. The sparse structure of the observations is therefore exploited by sparse optimization techniques, which reduce computational cost and memory usage.