Bayesian Classifiers

Bayes' theorem, applied to Computer Vision, is a fundamental technique for pattern classification based on experience (training set).

To understand Bayes' theorem, a simple example is useful. Suppose that we want to classify fruit shown to an observer (a computer in the limiting case). For simplicity, assume that there are only two types of fruit (the classifier categories), for example, oranges and apples. For humans, and likewise for machines, determining the type of fruit being observed is done by examining certain characteristics (features) extracted from the observation of the fruit using suitable techniques.

If the fruits are selected completely at random and no additional information can be extracted from them, the optimal approach to classifying them would be to provide a completely random answer.

Bayesian decision theory plays an important role only when some a priori information about the objects is known.

As a first step, suppose that we have no knowledge of what the fruits look like, but we know that 80% of the fruit consists of apples and the remaining 20% consists of oranges. If this is the only information on which to base the decision, one would instinctively classify the fruit as an apple (the optimal classifier): every fruit would be classified as an apple because, in the absence of other information, this is the only way to minimize the error. In this case, the a priori information consists of the probabilities that the selected fruit is an apple or an orange.

Now consider the case in which it is possible to extract some additional information from the observed scene. Bayes' concept applied to classification is also very intuitive from this perspective: if I observe a particular measurable characteristic of the image $x$ (features), I can estimate the probability that the image represents a given class $y_i$ a posteriori of the observation. From this perspective, Bayesian classifiers provide exactly the probability that the input data vector represents the specified output class.