Fish-Eye Camera Model

By generalizing the concept of the pin-hole camera, it is possible to introduce several classes of camera models. It is important to remember that real cameras are never ideal: no real optical system conforms perfectly to a single projection model. For this reason, various basic models approximate the lens equation, but suitable distortion terms must still be added to them.

Fish-eye optics represent a special case in which barrel distortion is dominant; this makes it possible to obtain extremely wide fields of view for a given focal length, up to $180^\circ$ or even more.

Let $\vartheta$ be the angle of incidence of the optical ray from the camera point $(x,y,z)$ with respect to the optical axis $(0,0,1)$. This angle is $\vartheta = \arctan r_i$, obtained from

$$r_i = \sqrt{x_1^2 + y_1^2} = \frac{\sqrt{x^2 + y^2}}{z}$$ (9.79)

where $x_1 = x / z$ and $y_1 = y / z$. It should be noted that the following equality holds:

$$\vartheta = \arctan \frac{\sqrt{x^2 + y^2}}{z} = \arccos \frac{z}{\sqrt{x^2 + y^2 + z^2}}$$ (9.80)

The idea is to regard all camera models as mappings of the angle of incidence $\vartheta$ to an angle $\vartheta'$, or, more commonly, as a $r(\vartheta) = f \vartheta'$ transformation that maps the angle of the incident ray to a pixel radius.

The various fish-eye lenses follow slightly different equations. Among the ideal models (and therefore without distortion), we can identify

• linear: $r = f \vartheta$
• orthographic: $r = f \sin \vartheta$
• equisolid angle: $r = 2 f \sin (\vartheta/2)$
• stereographic: $r = 2 f \tan ( \vartheta/2)$.

The pin-hole model can be regarded as a special case, since the relationship between the angle of incidence of the light ray $\vartheta$ and the pixel coordinate follows the rule $r = f \tan \vartheta$.

By absorbing the focal length into $r$, which thus becomes a radius expressed directly in pixels, the preceding equations make it possible to project an optical ray onto the image plane as follows:

$$u = \frac{x}{\sqrt{x^2+y^2}} r + u_0 \qquad v = \frac{y}{\sqrt{x^2+y^2}} r + v_0$$ (9.81)

since the phase angle remains constant between the incident ray and the projected ray in the absence of tangential distortion.

The above class of ideal models does not account for possible nonlinearities in the optics. The Kannala–Brandt model (KHB09) generalizes these equations by parameterizing a generic fish-eye lens through the term $r(\vartheta)$, using the classical Taylor-series expansion, which makes it possible to incorporate both the different lens models presented above and any distortions introduced by the optics.

It is useful to recall that, also in fish-eye cameras, all optical rays converge at a single projection center; thus, a pin-hole still exists. However, all equations that can be expressed linearly in a pin-hole model using homogeneous coordinates are no longer linear in fish-eye models. Consequently, even when working with homogeneous coordinates, it is necessary to use nonlinear formulations, as will be discussed in the next chapter.