Requirements for Visual Perception of Automotive Environments

This chapter deals with vision-based surveillance systems installed on moving vehicles. Some important issues must be carefully considered in the design of a visual perception system for mobile automotive applications.

Firstly, these systems, when used for road and/or driver monitoring, require faster processing than other applications, since the vehicle speed is bounded by the processing rate, and, therefore, real-time performance constraints are to be strictly met.

Secondly, in general in the automotive field no assumptions can be made on key parameters, such as scene illumination or contrast, which are directly measured by the vision sensor. Hence, the subsequent processing must be robust enough to adapt to different environmental conditions (for example, sun, rain, fog). In addition to this, a moving vision system has to take into account dynamic and sometimes abrupt changes in the illumination or contrast (such as transitions between sun and shadow, or the entrance or exit from a tunnel). On the other hand, a static surveillance system may take advantage of a partial knowledge about the environment, a fixed background, and slowly varying conditions.

Furthermore, when the acquisition system is installed on a moving vehicle other key issues, such as the robustness to vehicle's movements and drifts in the camera's calibration, must be handled. Image stabilization is a critical problem as well.

Eventually, on-board perception systems are obviously safety critical devices, requiring a strong degree of reliability and tolerance both to hardware and software failures.

However, recent advances in both computer and sensor technologies promote the use of machine vision on-board of intelligent vehicles. The developments in computational hardware, such as a higher degree of integration and a reduction of the power supply voltage, permit to produce machines that can deliver a high computing power, with fast networking facilities at an affordable price. In addition to this, current cameras include new important features that permit the handling and solving of some basic problems directly at sensor level. The resolution of the sensors has been drastically enhanced and the camera dynamics extended, and in order to decrease the image acquisition and transfer time, new technological solutions can be found in CMOS sensors, such as the possibility of dealing with pixels independently.