Deep Learning

Neural networks, and in particular their training through backpropagation, present several practical challenges:

• they require labeled training data, whereas in contexts involving large amounts of data (big data), most data may be uncategorized;
• training time scales poorly both as the network size increases and as the amount of data increases;
• optimization may become stuck in local minima, making the network suboptimal;
• supervised training of deep models (networks with many hidden layers) is a particularly complex optimization problem.

To address these challenges, a branch of machine learning known as deep learning has emerged. It exploits deep architectures and advanced techniques to improve learning effectiveness.

Humans address complex problems by dividing them into subproblems and multiple levels of abstract representation. Similarly, deep learning enables a system to learn hierarchical representations of data, directly mapping complex functions between input and output without relying on manually designed features.

This approach makes it possible to generate high-level abstractions that are often difficult for humans to describe explicitly but are more manageable for a computer.

As the availability of data and the number of machine learning applications increase, machine-learning techniques are evolving rapidly. The goal of deep learning is to construct high-level representations of data through multiple layers of nonlinear operations, as in Deep Neural Networks (DNNs).