Fish-Eye Camera Model

To generalize the concept of a pinhole camera, multiple classes of camera models can be introduced. It is important to note that real cameras are never ideal cameras that perfectly conform to a specific optical model. Therefore, there are various basic models that attempt to approximate the lens equation, to which several distortion terms must still be added.

In particular, fish-eye lenses are characterized by a dominant barrel distortion, which allows for achieving very high field of view angles, up to 180 degrees or more, while maintaining the same focal length.

Let $\vartheta$ be the incident angle of the optical ray at the camera point $(x,y,z)$ with respect to the optical axis $(0,0,1)$. This angle is given by $\vartheta = \arctan r_i$, which is derived from

$$r_i = \sqrt{x_1^2 + y_1^2} = \frac{\sqrt{x^2 + y^2}}{z}$$ (8.69)

, where indeed $x_1 = x / z$ and $y_1 = y / z$. Note that the equality holds as

$$\vartheta = \arctan \frac{\sqrt{x^2 + y^2}}{z} = \arccos \frac{z}{\sqrt{x^2 + y^2 + z^2}}$$ (8.70)

.

The idea is to consider all camera models as a manipulation of the incident angle $\vartheta$ into an angle $\vartheta'$, or it is often referred to as a transformation $r(\vartheta) = f \vartheta'$ that converts the angle of the incident ray into a ray in pixel coordinates.

The various fish-eye lenses follow slightly different equations, among which we can highlight the ideal models (thus without distortion):

• linear: $r = f \vartheta$
• orthographic: $r = f \sin \vartheta$
• constant solid angle: $r = 2 f \sin (\vartheta/2)$
• stereographic: $r = 2 f \tan ( \vartheta/2)$.

The pin-hole model can be viewed as a particular case since the ratio between the incident light angle $\vartheta$ and the pixel coordinates follows the rule $r = f \tan \vartheta$.

Having absorbed in $r$ also the focal length (thus it is a ray in pixels), the equations above allow us to transform the angle of incident light on the optics into a ray projected onto the sensor:

$$u = \frac{x}{\sqrt{x^2+y^2}} r + u_0 \qquad v = \frac{y}{\sqrt{x^2+y^2}} r + v_0$$ (8.71)

since the phase angle remains constant between the incident ray and the projected ray in the absence of tangential distortion.

The class of ideal models above does not account for potential nonlinearities in optics. The Kannala-Brandt model (KHB09) generalizes these equations by parameterizing a generic fish-eye lens in the term $r(\vartheta)$ using the classical Taylor series expansion, which allows for the incorporation of both the various lens models presented and any distortions introduced by the optics.

I recall that even in fish-eye cameras, all optical rays converge at the same point (therefore, the rays always pass through a pin-hole). However, all the equations that can be formulated (and several will be seen in the next chapter) in linear form using homogeneous coordinates with a fish-eye model as listed here can no longer be applied. We must transition to non-linear equations, even in homogeneous coordinates.