Lens Distortion

The majority of commercial cameras deviate from the pin-hole camera model, and this deviation is generally greater the larger the camera's field of view. Since each optical system consists of a certain number of lenses, the distortion arises from imperfections during the production and assembly phases of the optics. Achieving a distortion-free lens is an extremely costly process, and particularly in low-cost applications where reliance on economical optics is necessary, this issue becomes very evident.

These non-idealities generate a nonlinear distortion that is challenging to model, and due to the fact that this distortion depends on the interaction between the lens and the sensor, lens manufacturers typically do not provide, or are unable to provide, geometric information on how to represent such distortion.

It is important to note that the pin-hole camera model is only valid if the image being processed is not distorted; therefore, calibration, or correcting for geometric distortion, is a prerequisite for accurately reconstructing the three-dimensionality of the observed scene.

From the perspective of the optical ray, the distortion introduced by the lens is situated between the world and the pin-hole. The equation of the pin-hole camera modified with the optical distortion transforms into

$$\mathbf{p} = \mathbf{K} f_d( [\mathbf{Rt}]\mathbf{x} )$$ (8.7)

With this formalism, the distortion $f_d$ transforms a point from undistorted coordinates to distorted coordinates. This choice, as opposed to the inverse formulation, arises from purely practical considerations: Since the objective is to obtain a dense and undistorted output image (see the discussion in section 1.12), it is necessary to compute the function that transforms a non-distorted point into a distorted point.

In general, the distorting contributions of the lens are divided into radial (directly along the line connecting the point to the center of distortion) or tangential (which are perpendicular to the line). Tangential contributions (and other contributions not mentioned here) are typically small, while radial distortion is always detectable and generally increases in intensity as the focal length becomes shorter.

This section focuses on deriving a general relationship between the ideal point $(x,y)$ and the actual distorted image point observed $(\breve{x},\breve{y})$.

In the entire image, there exists a single point $(x_d,y_d)$, defined as the distortion center, where distortion does not produce any effects. For this point $(x,y)=(\breve{x},\breve{y})$.

To define the distortion, it is necessary to operate in a new set of coordinates, relative to the center of distortion:

$$\begin{array}{l} \bar{x} = x - x_d \\ \bar{y} = y - y_d \\ \end{array}$$ (8.8)

The distortion center is typically located near $(0,0)$, but there is no guarantee that it coincides with the principal point. In several articles, it is proposed as an approximation to ignore the distortion center and align it with the principal point or to consider only the term of decentering distortion.

The classical formulation of Brown-Conrady (Bro66) models lens distortion in the form of a deviation:

$$\begin{array}{l} \breve{x} = x + \delta_x (\bar{x},\bar{y}) \\ \breve{y} = y + \delta_y (\bar{x},\bar{y}) \end{array}$$ (8.9)

Such deviations can be divided into contributions:

radial distortion

The deviation due to radial distortion is described by the equation

$$\begin{array}{l} \delta^{r}_x = \bar{x} f_r ( r ) \\ \delta^{r}_y = \bar{y} f_r ( r ) \\ \end{array}$$ (8.10)

where $f_r(r)$ is a function solely of the radius $r = \sqrt{\bar{x}^{2}+\bar{y}^{2} }$, the Euclidean distance between the point and the center of distortion, subject to the constraint $f_r (0) = 1$.

The function $f_r(r)$ of radial distortion is not a well-known model but can be approximated using the first terms of the series expansion: The presence of only the powers of 2 is due to the symmetry of the function $f_r$.

thin prism distortion

imperfections in the manufacturing process and misalignment between the sensor and the lens introduce additional asymmetric distortions. This is typically modeled as It seems that you have entered a placeholder or a command related to a mathematical block. Please provide the content or text you would like to have translated, and I will be happy to assist you!

Such contributions are often inadequate, however, to describe the effects of optical decentralization.

decentering distortion

It is typically caused by improper assembly of the lens and the various components that make up the optics. The Brown-Conrady model represents the contribution of decentering in the form of

$$\begin{array}{l} \delta^{(t)}_x = (p_1 (r^2 + 2 \bar{x}^2) +... ...2) + 2 p_1 \bar{x} \bar{y} ) (1 + p_3 r^2 + \ldots) \end{array}$$ (8.11)

This contribution consists of both a radial component and a tangential component.

Incorporating all these contributions into equation (8.9), the overall Brown-Conrady model can be expressed as In fact, the radial distortion is dominant, and in most applications, the first terms are more than sufficient.

For example, OpenCV models distortion using the R3P1 model: 3 radial terms ($k_1$, $k_2$, $k_3$) and the first-degree decentering term ($p_1$, $p_2$).

The distortion coefficients are derived using various techniques available in the literature, applied to images acquired in a structured environment (calibration grids). Typically, a nonlinear minimizer is employed, and one either works with lines and iterates until all the curves in the image become straight plumb-line method (DF01), or enforces that points on a plane with known coordinates represent a homography. Such techniques are applicable only when operating in image coordinates (approach 1).

To calibrate the distortion in normalized camera coordinates (approach 2), both distortion and intrinsic camera parameters must be computed simultaneously (Zha99). An initial estimate of the intrinsic parameters can be obtained from a linear minimizer, but the final estimate is achieved only through a nonlinear minimizer.

The widely used technique for estimating distortion parameters involves optimizing the observation of feature points in the image, whose positions in world coordinates are known, in order to enforce a complete perspective projection (section 8.5.6).