Naive Bayes

Normally, a single feature extracted from the object to be classified is not sufficient to achieve high classification accuracy. Fortunately, many different features can be extracted from an image.

Let these features be denoted by $x_j$ and $j=1,\ldots,m$. It is important to note that the observed events $x_j$ used to construct the Bayesian classifier must be independent events (conditional independence); otherwise, Bayes' theorem is no longer valid, which is one of the limitations of Bayesian classifiers. For example, classifiers analyzing overlapping parts of an image cannot be combined, nor can the estimator “is orange” be combined with “is not red”.

The Naive Bayes (or idiot Bayes) assumption relies on the simplifying hypothesis that the observed attributes (features) are independent: in this case, given $m$ observed variables $x_1 \ldots x_m$, the probability that event $y_i$ occurs is:

$$p( x_1 \ldots x_m \vert y_i) = \prod^{m}_{j=1} p(x_j \vert y_i)$$ (5.12)