Naive Bayes

Typically, using a single feature extracted from the object to be classified does not yield a high classification accuracy. Fortunately, the features that can be extracted from an image are numerous.

Let the characteristics be indicated by $x_j$ and $j=1,\ldots,m$. It is very important to note that the observed events $x_j$ used to construct the Bayesian classifier must be independent events (conditional independence); otherwise, Bayes' theorem is no longer valid (one of the limitations of Bayesian classifiers): for example, classifiers analyzing overlapping parts of the image cannot be combined, nor can the estimator "is orange" be combined with "is not red."

The Naive Bayes (or idiot Bayes) assumption leverages the simplifying hypothesis of independence among the observed attributes (features): in this case, given $m$ observed variables $x_1 \ldots x_m$, the probability that the event $y_i$ occurs will be:

$$p( x_1 \ldots x_m \vert y_i) = \prod^{m}_{j=1} p(x_j \vert y_i)$$ (4.12)