liveBook · Manning

Deep learning

As explained in Grokking Machine Learning, Second Edition

A field of machine learning that uses neural networks.

Deep learning is the part of machine learning that uses neural networks. Neural networks are especially useful when simple models cannot capture a complicated pattern, such as recognizing images or processing text. Think of artificial intelligence as vehicles, machine learning as cars, and deep learning as a high-performance car within that larger group. section 1.7 section 10.0

Figure 1.3 Deep learning is a part of machine learning.

Use this diagram to place deep learning inside machine learning, which itself sits inside artificial intelligence. section 1.7

A neural network is a collection of perceptrons arranged in layers. Each perceptron combines input values with weights and a bias, then applies an activation function. Outputs from one layer become inputs to the next, allowing several simple classifiers to combine into a more complex, nonlinear classifier. section 10.5 section 10.6

Finding a pattern that one line misses

Suppose alien sentences contain the words “aack” and “beep,” and the goal is to classify the speaker as happy or sad. A single linear boundary cannot separate the two moods, but combining perceptrons can create a boundary made from multiple lines or a curve. This is the key intuition behind a neural network: layers can represent patterns that one perceptron cannot. section 10.1 section 10.7

Training adjusts the network so its predictions improve:

  • Start with randomly initialized weights and biases, then define an error function to measure performance. section 10.9
  • Repeatedly update weights and biases to reduce error; backpropagation uses loss derivatives to calculate those updates. section 10.25
  • Use activation functions such as ReLU and softmax; dropout and regularization can help address overfitting. section 10.25

Neural networks handle classification and regression. For a two-feature dataset with a circular class pattern, a Keras network can use ReLU hidden layers, dropout, and a softmax output to learn a nonlinear boundary. For handwritten digits, reshape each 28-by-28 image into a 784-value vector, convert labels to category vectors, train, and inspect both correct and incorrect predictions. section 10.18 section 10.19

Architectures can be changed to fit a dataset and problem, which makes neural networks versatile. Their applications include image recognition, text processing, medicine, self-driving cars, and, with further changes, generating text and images. section 10.21 section 1.13

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Builds from perceptrons and layered boundaries through backpropagation, overfitting, and Keras implementations.
Examples
Uses alien sentiment data, a circular two-feature dataset, and MNIST handwritten digits.
What it assumes
Builds neural networks from perceptrons while drawing on prior linear and logistic classifier ideas.
What you can do after
Apply it via Keras to train neural-network classifiers on nonlinear data and handwritten images.
How it connects
Connects neural networks with perceptrons, activation functions, backpropagation, regularization, dropout, classification, and regression.

Also explained in

Compare how 20 books explain deep learning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.