Naive Bayes

Normally, a single feature extracted from the object to be classified is not sufficient to achieve high classification accuracy. Fortunately, many different features can be extracted from an image.

Let $x_j$ denote these features, with $j=1,\ldots,m$. It is very important to note that the observed events $x_j$ used to construct the Bayesian classifier must be independent events (conditional independence); otherwise, Bayes' theorem is no longer valid (one of the limitations of Bayesian classifiers): for example, classifiers that analyze parts of the image having regions in common cannot be combined, nor can the estimator “is orange” be combined with “is not red”.

The Naive Bayes (or idiot Bayes) assumption exploits the simplifying hypothesis that the observed attributes (features) are independent: in this case, given $m$ observed variables $x_1 \ldots x_m$, the probability that event $y_i$ occurs is:

$$p( x_1 \ldots x_m \vert y_i) = \prod^{m}_{j=1} p(x_j \vert y_i)$$ (5.12)