Generalizing Training

Throughout this chapter, numerous classification models and different training strategies have been introduced. Although the algorithms differ considerably, they all share the same objective: to achieve good performance not only on the training data but also on previously unseen samples. This problem is known as generalization.

It is important to note that Machine Learning techniques involve much more than mere optimization. One of the objectives pursued during training is to ensure that the system can classify new samples that it has not yet seen. One way to combat overfitting is through “regularization”. Several regularization techniques are described in the literature; the main ones are L1/L2 regularization and early stopping (early-stopping).