Bayesian Classifiers

Bayes' theorem, when applied to Artificial Vision, is a fundamental technique for pattern classification based on experience (training set).

To understand Bayes' theorem, a simple example is useful. Suppose that we want to classify fruit shown to an observer (a computer in the extreme case). For simplicity, suppose that there are only two types of fruit (the classifier categories), for example, oranges and apples. For humans, and likewise for machines, determining the type of fruit being observed requires examining specific characteristics (features) extracted from the observation of the fruit through suitable techniques.

If the fruits are selected completely at random and no other information can be extracted from them, the optimal classification approach would be to provide a completely random answer.

Bayesian decision theory plays an important role only when some a priori information about the objects is known.

As a first step, suppose that we have no knowledge of what the fruits look like, but we know that 80% of the fruit consists of apples and the remainder consists of oranges. If this is the only information on which to base the decision, one would instinctively classify the fruit as an apple (the optimal classifier): each fruit will be classified as an apple because, in the absence of other information, this is the only way to minimize the error. In this case, the a priori information consists of the probabilities that the selected fruit is an apple or an orange.

Let us now consider the case in which it is possible to extract some additional information from the observed scene. Bayes' concept applied to classification is also very intuitive from this perspective: if I observe a particular measurable characteristic of the image $x$ (features), I can estimate the probability that the image represents a certain class $y_i$ a posteriori to the observation. From this perspective, Bayesian classifiers provide precisely the probability that the input data vector represents the given output class.