Representation Learning

Feature Learning, or Representation Learning, provides a bridge between traditional machine learning techniques and deep learning. The idea is to use unsupervised learning techniques to reduce the dimensionality of the problem while preserving as much information as possible, and then use the transformed data for supervised classification.

The performance of a machine learning algorithm depends strongly on the representation of the input data. Much of the effort is devoted to designing transformations that convert raw data into a format optimal for classification.

Useful unsupervised learning techniques for representation include:

• K-means segmentation
• Hopfield networks
• Sparse Coding
• Principal Component Analysis (PCA, Section 2.9.1)
• Restricted Boltzmann Machines (RBM, Section 5.10.2)
• AutoEncoder (Section 5.10.3)

These techniques aim to reduce dimensionality while preserving the most relevant information, namely, the information that best describes the training samples.