What does section 4.1.1, The IMDb dataset, cover?

This section introduces the IMDb movie-review dataset, its integer-encoded reviews and labels, and the reason for limiting its vocabulary. Describe the size, training and test split, and class balance of the IMDb dataset used here. Explain what the loaded review and label arrays contain and how the reviews are represented. Explain what num_words=10000 retains and why the section limits the vocabulary.

What does section 4.1.2, Preparing the data, cover?

The section prepares IMDb review sequences and labels as tensors that can be fed into a neural network. Explain why the integer lists need to be converted to tensors, and describe the two options given: padding them to a common length or multi-hot encoding them. Describe how the chosen multi-hot encoding represents a review. Describe the label preparation step performed after vectorizing the review inputs.

What does section 4.1.3, Building your model, cover?

This section builds and configures a Dense-layer model for predicting whether a movie review is positive. Describe the model chosen for review vectors with binary labels: two intermediate Dense layers of 16 units each, followed by a layer that outputs one sentiment prediction. Explain what choosing 16 Dense-layer units means for the representation and why increasing the number of units has both benefits and costs. Explain why the intermediate Dense layers use an activation such as relu rather than stacking layers made only of dot products and additions.

What does section 4.1.4, Validating your approach, cover?

The section uses a validation set to monitor an IMDb classifier, identify overfitting, choose a training duration, and preserve the test set for final evaluation. Explain why the model is monitored on held-out validation data rather than judged on its training data or repeatedly evaluated on the test set. Distinguish validation data used to guide training choices from unseen test data reserved for final evaluation. Describe how the example monitors loss and accuracy during training: it holds out 10,000 samples, supplies them to model.fit() as validation_data, and reads training and validation metrics from the returned History object. State that validation_split is an alternative way to hold out a fraction of the training arrays. Explain why improving training metrics do not imply improving performance on unseen data in the plotted results. Describe how the validation trend motivates training a fresh model for four epochs before evaluating it on the test set.

What does section 4.1.6, Further experiments, cover?

The section proposes controlled experiments to test whether the IMDb classifier’s architecture and training choices are reasonable. Describe how to test the classifier’s choices by varying its representation layers, units, loss function, or activation and checking the effect on validation and test accuracy.

What does section 4.1.7, Wrapping up, cover?

The section summarizes the IMDb example’s practical lessons about data preparation, model design, training choices, and monitoring overfitting. Explain why raw text must be prepared before a neural network can use it, and give the section’s example of an encoding. Describe the kind of intermediate layers the section identifies as useful for sentiment classification and many other problems. Explain the recommended output layer and loss for a two-class classification problem, including what the output represents.

What does section 4.2.1, The Reuters dataset, cover?

Introduces the Reuters newswire dataset and explains how its multiclass labels and text examples are represented in Keras. Describe the Reuters dataset as short newswires classified into 46 topics, with each label represented by an integer from 0 to 45. State that topics are unevenly represented, though every topic has at least 10 training examples. Explain that Keras provides the Reuters data as lists of integer word indices and that num_words=10000 limits it to the 10,000 most frequent words. State that the dataset has 8,982 training examples and 2,246 test examples.

What does section 4.2.2, Preparing the data, cover?

This section reuses the earlier input-vectorization approach and introduces integer and one-hot representations for multiclass labels. Describe the two ways to represent the labels—leave them as integers or one-hot encode them—and explain what a one-hot-encoded label looks like.

What does section 4.2.3, Building your model, cover?

This section explains how the Reuters model’s layer sizes, output, loss, and metrics suit a 46-class classification task. Explain why the model uses 64-unit intermediate Dense layers instead of the 16-dimensional layers used in the earlier example. Describe what the 46-unit softmax output layer produces for each input sample. Explain why categorical_crossentropy is used to train this model.

What should I know before reading chapter 4?

First steps with Keras (chapter 3): Keras is the framework used to load and work with the dataset.; Layers: The building blocks of deep learning (chapter 3): Dense layers as model building blocks; Tensor product (chapter 2): The dot product used in the Dense-layer projection; The gears of neural networks: Tensor operations (chapter 2): Tensor operations used within neural-network layers; The “compile” step: Configuring the learning process (chapter 3): Configuring a model’s learning process; Monitoring loss and metrics on validation data (chapter 3): Monitoring metrics on validation data during training.; Understanding the fit method (chapter 3): Using fit() to train a Keras model.; Inference: Using a model after training (chapter 3): Introduces using a model for inference after training.

Chapter 4: Classification and regression

What you will learn in chapter 4 of Deep Learning with Python, Third Edition

The chapter introduces classification and regression tasks and defines the vocabulary used to describe their inputs, outputs, and targets.

After this chapter you will be able to

Inputs predictions and targets. Explain how a sample, prediction, target, and prediction error relate, and what ground truth means for a dataset.
Classification labels and task types. Distinguish classes from labels and explain how binary, categorical, and multilabel classification differ.
Scalar and vector regression. Distinguish scalar regression from vector regression using the kinds of targets each predicts.
Binary movie review classification. Explain that binary classification chooses between two classes and that this example uses review text to classify a movie review as positive or negative.
Encoded Reviews and Labels. Explain what the loaded review and label arrays contain and how the reviews are represented.
Frequent Word Vocabulary Limit. Explain what num_words=10000 retains and why the section limits the vocabulary.
Fixed-shape review inputs. Explain why the integer lists need to be converted to tensors, and describe the two options given: padding them to a common length or multi-hot encoding them. Describe how the chosen multi-hot encoding represents a review.
Binary sentiment model architecture. Describe the model chosen for review vectors with binary labels: two intermediate Dense layers of 16 units each, followed by a layer that outputs one sentiment prediction.
Nonlinear activation purpose. Explain why the intermediate Dense layers use an activation such as relu rather than stacking layers made only of dot products and additions.
Sigmoid probability output. Describe why the final layer uses sigmoid and what its single output means for a review.
Binary model training configuration. Justify the choice of binary_crossentropy for this probability-producing binary classifier, and state the optimizer and metric selected when configuring it.
Validation and test set roles. Explain why the model is monitored on held-out validation data rather than judged on its training data or repeatedly evaluated on the test set. Distinguish validation data used to guide training choices from unseen test data reserved for final evaluation.
Overfitting and epoch choice. Explain why improving training metrics do not imply improving performance on unseen data in the plotted results. Describe how the validation trend motivates training a fresh model for four epochs before evaluating it on the test set.
Positive Review Predictions. Explain that the trained model’s predict method produces likelihoods of reviews being positive, and that outputs near 0 or 1 show greater confidence than outputs such as 0.4 or 0.6.
Classifier choice experiments. Describe how to test the classifier’s choices by varying its representation layers, units, loss function, or activation and checking the effect on validation and test accuracy.
Text preprocessing for networks. Explain why raw text must be prepared before a neural network can use it, and give the section’s example of an encoding.
Binary classifier output and loss. Explain the recommended output layer and loss for a two-class classification problem, including what the output represents.
Overfitting and held-out monitoring. Explain why improving training performance is not enough and what performance should also be monitored.
Single-label versus multilabel classification. Distinguish single-label multiclass classification from multilabel multiclass classification using the Reuters newswire example.
Reuters topics and labels. Describe the Reuters dataset as short newswires classified into 46 topics, with each label represented by an integer from 0 to 45. State that topics are unevenly represented, though every topic has at least 10 training examples.
Multiclass label representations. Describe the two ways to represent the labels—leave them as integers or one-hot encode them—and explain what a one-hot-encoded label looks like.
Intermediate layer information bottlenecks. Explain why the model uses 64-unit intermediate Dense layers instead of the 16-dimensional layers used in the earlier example.
Softmax class probabilities. Describe what the 46-unit softmax output layer produces for each input sample.
Categorical crossentropy training loss. Explain why categorical_crossentropy is used to train this model.
Validation-guided training duration. Explain how the validation results led to the choice of nine training epochs before test evaluation.
Empirical random accuracy baseline. Explain why the section checks a random classifier’s accuracy and how that result puts the model’s accuracy in context.
Topic probability predictions. Explain that calling predict on new samples returns, for each sample, a 46-entry probability distribution over topics, and that the topic with the highest probability is the predicted class.
Integer labels and loss choice. Explain that keeping labels as integer tensors calls for sparse_categorical_crossentropy instead of categorical_crossentropy, which expects categorically encoded labels. State that the two losses are mathematically the same but have different interfaces.
Intermediate layer information bottleneck. Explain why making an intermediate layer much smaller than the 46-dimensional output hurts classification, using the four-unit example and its validation result.
Intermediate layer configuration experiments. Describe experiments that change intermediate layer size and the number of intermediate layers before the final softmax classification layer.
Multiclass output layer design. Explain why a classifier for N classes ends with a Dense layer of size N and, for single-label multiclass classification, uses softmax to output a probability distribution over those classes.
Label encoding and loss pairing. Distinguish the two supported label-and-loss pairings, and explain why categorical crossentropy suits single-label multiclass classification.
Regression versus classification targets. Distinguish regression, which predicts a continuous value, from classification, which predicts a discrete label, using the section’s examples.
District features and target. Describe what one dataset sample represents, what information its eight numerical features provide, and what the model is meant to predict.
Feature-wise input normalization. Explain why inputs with widely different ranges make learning harder and describe how each input feature is normalized.
Training-only normalization statistics. State where the quantities for normalizing test inputs must come from and why test-data quantities must not be used.
House-price target scaling. Explain why the house-price targets are scaled, how they are scaled, and how predictions return to dollar values.
Small model for limited data. Justify using two intermediate layers of 64 units each when few training samples are available.
Unrestricted scalar regression output. Explain why the model ends in one unit without an activation function.
Small Data Validation Variance. Explain why a single training-validation split is unreliable when the house-price dataset has few examples.
K-Fold Validation Scores. Describe how K-fold cross-validation trains and evaluates models, and explain why this section averages the fold scores.
Epoch Choice From Validation. Explain how per-epoch validation MAE across folds is used to choose a training duration in this example.
Regression prediction meaning. Distinguish the regression model’s price prediction from the outputs of the binary and multiclass classification models. Explain that predict() gives a guessed price in hundreds of thousands of dollars for each sample.
Regression loss and evaluation metrics. Distinguish regression loss and evaluation metrics from those used for classification: MSE is a common regression loss, MAE is a common regression metric, and accuracy does not apply to regression.
Small dataset modeling choices. Justify using K-fold validation to evaluate a model reliably when data is scarce, and a small model with only one or two intermediate layers to avoid severe overfitting when training data is scarce.

Cover of Deep Learning with Python, Third Edition

liveBook manning.com

Read chapter 4 on liveBook Get the book at manning.com

What it builds on

First steps with Keras (chapter 3) · Keras is the framework used to load and work with the dataset.
Layers: The building blocks of deep learning (chapter 3) · Dense layers as model building blocks
Tensor product (chapter 2) · The dot product used in the Dense-layer projection
The gears of neural networks: Tensor operations (chapter 2) · Tensor operations used within neural-network layers
The “compile” step: Configuring the learning process (chapter 3) · Configuring a model’s learning process
Monitoring loss and metrics on validation data (chapter 3) · Monitoring metrics on validation data during training.
Understanding the fit method (chapter 3) · Using fit() to train a Keras model.
Inference: Using a model after training (chapter 3) · Introduces using a model for inference after training.

Section by section

Introduction

The chapter introduces classification and regression tasks and defines the vocabulary used to describe their inputs, outputs, and targets.

Inputs predictions and targets: Explain how a sample, prediction, target, and prediction error relate, and what ground truth means for a dataset.
Why it matters: These terms let you describe what a model is given, what it produces, and how its output is judged.
Classification labels and task types: Distinguish classes from labels and explain how binary, categorical, and multilabel classification differ.
Why it matters: The distinction tells you whether a task asks for one category or potentially several for each sample.
Scalar and vector regression: Distinguish scalar regression from vector regression using the kinds of targets each predicts.
Why it matters: The number of continuous target values determines which kind of regression problem you are describing.

Key terms: sample, input, prediction, output, target, prediction error, classes, label, binary classification, categorical classification

4.1 Classifying movie reviews: A binary classification example

Introduces binary classification through the task of labeling movie reviews positive or negative from their text.

Binary movie review classification: Explain that binary classification chooses between two classes and that this example uses review text to classify a movie review as positive or negative.
Why it matters: This identifies what the model is being asked to predict before the example begins.

Key terms: two-class classification, binary classification, positive, negative

4.1.1 The IMDb dataset

This section introduces the IMDb movie-review dataset, its integer-encoded reviews and labels, and the reason for limiting its vocabulary.

IMDb Review Split: Describe the size, training and test split, and class balance of the IMDb dataset used here.
Why it matters: The split and label balance establish what data the classification example starts with.
Encoded Reviews and Labels: Explain what the loaded review and label arrays contain and how the reviews are represented.
Why it matters: The model receives encoded reviews and numeric labels rather than raw review text.
Frequent Word Vocabulary Limit: Explain what num_words=10000 retains and why the section limits the vocabulary.
Why it matters: The vocabulary limit makes the encoded reviews practical to work with without retaining largely unhelpful rare words.

Key terms: IMDb dataset, training, testing, negative, positive, preprocessed, word indices, train_data, test_data, train_labels

4.1.2 Preparing the data

The section prepares IMDb review sequences and labels as tensors that can be fed into a neural network.

Fixed-shape review inputs: Explain why the integer lists need to be converted to tensors, and describe the two options given: padding them to a common length or multi-hot encoding them. Describe how the chosen multi-hot encoding represents a review.
Why it matters: This explains why the reviews must be reshaped before the model can use them.
Float-valued review labels: Describe the label preparation step performed after vectorizing the review inputs.
Why it matters: Preparing the labels as well as the inputs completes the data preparation described here.

Key terms: integer sequences, tensors, multi-hot encode, Embedding layer, labels, NumPy arrays, ints, floats

4.1.3 Building your model

This section builds and configures a Dense-layer model for predicting whether a movie review is positive.

Binary sentiment model architecture: Describe the model chosen for review vectors with binary labels: two intermediate Dense layers of 16 units each, followed by a layer that outputs one sentiment prediction.
Why it matters: The layer layout connects the review data to the single prediction the model must make.
Layer width and representation capacity: Explain what choosing 16 Dense-layer units means for the representation and why increasing the number of units has both benefits and costs.
Why it matters: Layer width is a choice about both what the model can represent and what it may learn unnecessarily.
Nonlinear activation purpose: Explain why the intermediate Dense layers use an activation such as relu rather than stacking layers made only of dot products and additions.
Why it matters: Nonlinearity is what lets the stacked layers represent more than a single linear transformation.
Sigmoid probability output: Describe why the final layer uses sigmoid and what its single output means for a review.
Why it matters: The output activation makes the model’s sentiment score interpretable as a probability.
Binary model training configuration: Justify the choice of binary_crossentropy for this probability-producing binary classifier, and state the optimizer and metric selected when configuring it.
Why it matters: The loss and metric should match the binary probability prediction the model makes.

Key terms: Dense layers, intermediate layers, units, scalar prediction, weight matrix W, representation space, relu, activation function, non-linearity, hypothesis space

4.1.4 Validating your approach

The section uses a validation set to monitor an IMDb classifier, identify overfitting, choose a training duration, and preserve the test set for final evaluation.

Validation and test set roles: Explain why the model is monitored on held-out validation data rather than judged on its training data or repeatedly evaluated on the test set. Distinguish validation data used to guide training choices from unseen test data reserved for final evaluation.
Why it matters: Keeping the test set untouched gives you a more trustworthy final measure of performance on new data.
Training and validation monitoring: Describe how the example monitors loss and accuracy during training: it holds out 10,000 samples, supplies them to model.fit() as validation_data, and reads training and validation metrics from the returned History object. State that validation_split is an alternative way to hold out a fraction of the training arrays.
Why it matters: The recorded curves let you compare improvement on training samples with performance on held-out samples.
Overfitting and epoch choice: Explain why improving training metrics do not imply improving performance on unseen data in the plotted results. Describe how the validation trend motivates training a fresh model for four epochs before evaluating it on the test set.
Why it matters: Validation trends can reveal when more training is making the model worse on unseen examples.

Key terms: validation set, test set, training data, validation_data, validation_split, model.fit(), History object, history, loss, training loss

4.1.5 Using a trained model to generate predictions on new data

This section shows how to use a trained model to predict the likelihood that a review is positive and interpret its confidence.

Positive Review Predictions: Explain that the trained model’s predict method produces likelihoods of reviews being positive, and that outputs near 0 or 1 show greater confidence than outputs such as 0.4 or 0.6.
Why it matters: It helps you use the model on reviews and read its predictions without treating every output as equally certain.

Key terms: predict method, likelihood, positive, confident

4.1.6 Further experiments

The section proposes controlled experiments to test whether the IMDb classifier’s architecture and training choices are reasonable.

Classifier choice experiments: Describe how to test the classifier’s choices by varying its representation layers, units, loss function, or activation and checking the effect on validation and test accuracy.
Why it matters: Comparing alternatives helps you judge whether the original design is reasonable and where it might improve.

Key terms: representation layers, units, validation and test accuracy, mean_squared_error, binary_crossentropy, tanh

4.1.7 Wrapping up

The section summarizes the IMDb example’s practical lessons about data preparation, model design, training choices, and monitoring overfitting.

Text preprocessing for networks: Explain why raw text must be prepared before a neural network can use it, and give the section’s example of an encoding.
Why it matters: A network needs its input in a usable tensor representation.
Dense layers with relu: Describe the kind of intermediate layers the section identifies as useful for sentiment classification and many other problems.
Why it matters: This is a reusable model pattern, not a choice limited to movie reviews.
Binary classifier output and loss: Explain the recommended output layer and loss for a two-class classification problem, including what the output represents.
Why it matters: The output representation and loss need to match the two-class task.
Adam optimizer default: State the section’s practical recommendation about choosing an optimizer.
Why it matters: It gives you a practical starting choice without claiming optimizer choice never matters.
Overfitting and held-out monitoring: Explain why improving training performance is not enough and what performance should also be monitored.
Why it matters: Training results alone can hide worsening performance on new data.

Key terms: preprocessing, tensors, binary vectors, Dense layers, relu activations, sentiment classification, binary classification, Dense layer, sigmoid activation, binary_crossentropy

4.2 Classifying newswires: A multiclass classification example

The section frames Reuters topic classification as a single-label, multiclass problem and distinguishes it from multilabel classification.

Single-label versus multilabel classification: Distinguish single-label multiclass classification from multilabel multiclass classification using the Reuters newswire example.
Why it matters: The distinction tells you what kind of classification problem the Reuters example represents.

Key terms: multiclass classification, single-label, multiclass classification, multilabel, multiclass classification

4.2.1 The Reuters dataset

Introduces the Reuters newswire dataset and explains how its multiclass labels and text examples are represented in Keras.

Reuters topics and labels: Describe the Reuters dataset as short newswires classified into 46 topics, with each label represented by an integer from 0 to 45. State that topics are unevenly represented, though every topic has at least 10 training examples.
Why it matters: The number and meaning of the labels define the classification task.
Newswire input representation: Explain that Keras provides the Reuters data as lists of integer word indices and that num_words=10000 limits it to the 10,000 most frequent words. State that the dataset has 8,982 training examples and 2,246 test examples.
Why it matters: The input format and vocabulary limit explain what the model receives before data preparation.

Key terms: Reuters dataset, topics, topic index, training set, num_words=10000, word indices, training examples, test examples

4.2.2 Preparing the data

This section reuses the earlier input-vectorization approach and introduces integer and one-hot representations for multiclass labels.

Multiclass label representations: Describe the two ways to represent the labels—leave them as integers or one-hot encode them—and explain what a one-hot-encoded label looks like.
Why it matters: The label representation determines what form the classification targets take.

Key terms: labels, one-hot encoding, categorical encoding, label index

4.2.3 Building your model

This section explains how the Reuters model’s layer sizes, output, loss, and metrics suit a 46-class classification task.

Intermediate layer information bottlenecks: Explain why the model uses 64-unit intermediate Dense layers instead of the 16-dimensional layers used in the earlier example.
Why it matters: The number of classes affects how much information intermediate layers may need to preserve.
Softmax class probabilities: Describe what the 46-unit softmax output layer produces for each input sample.
Why it matters: The output gives a probability for every possible topic, rather than only a single class label.
Categorical crossentropy training loss: Explain why categorical_crossentropy is used to train this model.
Why it matters: The loss gives training a way to compare the model’s class probabilities with the labels.
Top three accuracy metric: Distinguish ordinary accuracy from top-k accuracy and explain why the model also monitors top-3 accuracy.
Why it matters: Top-3 accuracy reveals near misses that ordinary accuracy counts simply as wrong.

Key terms: Dense layers, intermediate layers, information bottleneck, 64 units, Dense layer of size 46, softmax activation, probability distribution, output[i], categorical_crossentropy, loss function

4.2.4 Validating your approach

The section uses validation curves to choose a training duration, then compares test accuracy with an empirical random baseline.

Validation-guided training duration: Explain how the validation results led to the choice of nine training epochs before test evaluation.
Why it matters: Validation curves help choose when to stop training before evaluating on the test set.
Empirical random accuracy baseline: Explain why the section checks a random classifier’s accuracy and how that result puts the model’s accuracy in context.
Why it matters: Accuracy is easier to interpret when compared with a baseline suited to the task.

Key terms: validation set, loss and accuracy curves, overfit, epochs, test set, accuracy, random baseline, random classifier, 46 classes

4.2.5 Generating predictions on new data

This section shows how to interpret a trained newswire classifier’s predictions on new samples.

Topic probability predictions: Explain that calling predict on new samples returns, for each sample, a 46-entry probability distribution over topics, and that the topic with the highest probability is the predicted class.
Why it matters: It lets you turn the model’s numeric output into a topic prediction.

Key terms: predict method, class probability distribution, predicted class

4.2.6 A different way to handle the labels and the loss

This section explains how to choose the loss function when multiclass labels are kept as integers rather than categorically encoded.

Integer labels and loss choice: Explain that keeping labels as integer tensors calls for sparse_categorical_crossentropy instead of categorical_crossentropy, which expects categorically encoded labels. State that the two losses are mathematically the same but have different interfaces.
Why it matters: Matching the loss interface to the label format lets you use integer labels without categorically encoding them.

Key terms: integer tensors, categorical encoding, categorical_crossentropy, sparse_categorical_crossentropy

4.2.7 The importance of having sufficiently large intermediate layers

This section shows how an undersized intermediate layer creates an information bottleneck in a 46-class classifier.

Intermediate layer information bottleneck: Explain why making an intermediate layer much smaller than the 46-dimensional output hurts classification, using the four-unit example and its validation result.
Why it matters: A narrow layer can limit a classifier even when the rest of the model is capable of learning the task.

Key terms: information bottleneck, intermediate layers, validation accuracy

4.2.8 Further experiments

This section suggests changing the width and number of intermediate layers to build intuition about model configuration choices.

Intermediate layer configuration experiments: Describe experiments that change intermediate layer size and the number of intermediate layers before the final softmax classification layer.
Why it matters: Trying these alternatives helps you develop intuition for configuring classification models.

Key terms: intermediate layers, units, softmax classification layer

4.2.9 Wrapping up

Summarizes how to configure a single-label multiclass classifier’s output, loss, label encoding, and intermediate layer sizes.

Multiclass output layer design: Explain why a classifier for N classes ends with a Dense layer of size N and, for single-label multiclass classification, uses softmax to output a probability distribution over those classes.
Why it matters: This connects the number of classes to the model’s output shape and interpretation.
Label encoding and loss pairing: Distinguish the two supported label-and-loss pairings, and explain why categorical crossentropy suits single-label multiclass classification.
Why it matters: The loss choice must match how the class labels are represented.
Intermediate layer information bottlenecks: Explain why intermediate layers that are too small should be avoided when classifying into many categories.
Why it matters: Layer sizes before the output can limit a multiclass model, not just the final layer size.

Key terms: Dense layer, softmax activation, probability distribution, single-label, multiclass classification, categorical crossentropy, categorical encoding, one-hot encoding, categorical_crossentropy, sparse_categorical_crossentropy, information bottlenecks

4.3 Predicting house prices: A regression example

This section introduces regression as predicting continuous values rather than discrete class labels and cautions that logistic regression is a classification algorithm.

Regression versus classification targets: Distinguish regression, which predicts a continuous value, from classification, which predicts a discrete label, using the section’s examples.
Why it matters: The prediction target tells you whether a problem is regression or classification.
Logistic regression naming exception: State that, despite its name, logistic regression is a classification algorithm rather than a regression algorithm.
Why it matters: Algorithm names do not always identify the type of prediction task.

Key terms: classification, regression, discrete label, continuous value, logistic regression, classification algorithm

4.3.1 The California Housing Price dataset

Introduces a small California housing dataset for predicting district-level median home prices from eight numerical features.

District features and target: Describe what one dataset sample represents, what information its eight numerical features provide, and what the model is meant to predict.
Why it matters: This identifies what the model receives as input and what its prediction means.
Small dataset and split: Explain why the example uses the small dataset version and state how many training and test samples it contains.
Why it matters: The small sample count explains why this example is useful practice for limited-data problems.

Key terms: block group, district, numerical features, median value, small version, training samples, test samples

4.3.2 Preparing the data

This section prepares house-price data for training by normalizing input features with training-set statistics and scaling target prices to a manageable range.

Feature-wise input normalization: Explain why inputs with widely different ranges make learning harder and describe how each input feature is normalized.
Why it matters: Putting features on comparable scales makes learning easier for the model.
Training-only normalization statistics: State where the quantities for normalizing test inputs must come from and why test-data quantities must not be used.
Why it matters: Using test-data statistics would let information from the test set enter the workflow.
House-price target scaling: Explain why the house-price targets are scaled, how they are scaled, and how predictions return to dollar values.
Why it matters: Scaling prices lets the model start learning without first having to reach very large output values.

Key terms: feature-wise normalization, mean, standard deviation, training data, test data, normalizing the test data, targets, weights, learning rate, predictions

4.3.3 Building your model

This section explains the model size, output layer, loss, and metric chosen for predicting a single continuous house price.

Small model for limited data: Justify using two intermediate layers of 64 units each when few training samples are available.
Why it matters: The model’s size is a response to the amount of training data, not an arbitrary choice.
Unrestricted scalar regression output: Explain why the model ends in one unit without an activation function.
Why it matters: The output layer must allow the range of values the regression target can take.
Mean squared error loss: Describe the mean_squared_error loss used to compile the regression model.
Why it matters: This identifies what prediction error the model is trained to reduce.
MAE in house-price units: Explain what the monitored mean absolute error measures and interpret an MAE of 0.5 using the target scaling.
Why it matters: Converting the metric back to dollars makes the reported prediction error interpretable.

Key terms: intermediate layers, units, overfitting, scalar regression, linear layer, activation function, sigmoid, mean_squared_error, mean squared error, loss function

4.3.4 Validating your approach using K-fold validation

This section uses K-fold validation to evaluate a house-price model with limited data, choose a training duration, and assess a final model.

Small Data Validation Variance: Explain why a single training-validation split is unreliable when the house-price dataset has few examples.
Why it matters: It explains why this example needs a more reliable validation approach.
K-Fold Validation Scores: Describe how K-fold cross-validation trains and evaluates models, and explain why this section averages the fold scores.
Why it matters: Averaging across folds makes the evaluation less dependent on one fortunate or unfortunate split.
Epoch Choice From Validation: Explain how per-epoch validation MAE across folds is used to choose a training duration in this example.
Why it matters: The validation curve gives a reason to stop training rather than choose the epoch count arbitrarily.
Final Model Evaluation: Describe what happens after tuning the model's parameters and how the final model is evaluated.
Why it matters: It separates parameter tuning from the final check of the trained model.

Key terms: validation set, validation split, high variance, K-fold cross-validation, partitions, folds, validation score, per-epoch validation score log, mean absolute error (MAE), overfitting

4.3.5 Generating predictions on new data

This section explains how to interpret predictions from the house-price regression model and contrasts them with classification predictions.

Regression prediction meaning: Distinguish the regression model’s price prediction from the outputs of the binary and multiclass classification models. Explain that predict() gives a guessed price in hundreds of thousands of dollars for each sample.
Why it matters: The output’s meaning and units determine how you interpret a prediction.

Key terms: predict(), scalar regression model, hundreds of thousands of dollars

4.3.6 Wrapping up

Summarizes the loss, metrics, preprocessing, validation, and model-size choices illustrated by the scalar regression example.

Regression loss and evaluation metrics: Distinguish regression loss and evaluation metrics from those used for classification: MSE is a common regression loss, MAE is a common regression metric, and accuracy does not apply to regression.
Why it matters: Choosing a loss and metric that suit the task makes a regression model’s training and evaluation meaningful.
Independent input feature scaling: Explain that input features with values in different ranges should each be scaled independently during preprocessing.
Why it matters: This gives you a preprocessing rule to apply when input features have different value ranges.
Small dataset modeling choices: Justify using K-fold validation to evaluate a model reliably when data is scarce, and a small model with only one or two intermediate layers to avoid severe overfitting when training data is scarce.
Why it matters: Scarce data affects both how reliably you can evaluate a model and how large a model you should train.

Key terms: scalar regression, mean squared error (MSE), MAE, accuracy, features, scaled independently, preprocessing, K-fold validation, intermediate layers, overfitting

4.4 Summary

Where this is used later

(chapter intro) (chapter 6) · The earlier worked examples begin with an available dataset.
Preparing the data (chapter 6) · The text classification example converted word-sequence data into tensors.
Increasing model capacity (chapter 5) · The idea of representation capacity informs discussion of increasing model capacity.
Training, validation, and test sets (chapter 5) · The distinction between validation and test sets underlies the later treatment of data splits.
Generalization: The goal of machine learning (chapter 5) · The earlier example's use of validation data to check a trained model.
Underfitting and overfitting (chapter 5) · The earlier classification example used held-out validation data to track model performance.
Using early stopping (chapter 5) · The four-epoch choice motivates the later discussion of stopping training to improve generalization.
Beating a common-sense baseline (chapter 5) · Judging model performance against a baseline.

Concepts explained in this chapter

classifierA supervised learning task that assigns text inputs to one of several classes.
evaluation datasetA held-out subset of the training data used during training to monitor performance and tune choices without touching the final test set.
linear regressionA transformation that preserves linear structure and can be represented by matrix multiplication.
linear layerA linear layer is a neural-network layer that maps input features to outputs by a learned affine transformation, typically connecting each output to every input feature.
sigmoidA statistical classification method that models class probabilities with a linear predictor passed through a logistic function.
overfittingThe failure mode where performance improves on training data while degrading on unseen data because the model has adapted too specifically to the training set.
cross entropy lossA classification loss that measures the mismatch between predicted probability distributions and true token labels.
activation functionA nonlinear function applied to a layer’s output to reshape it and increase model expressiveness.
ReLUA rectified linear activation operation applied element-wise to tensor entries, returning the input when positive and zero otherwise.
adamAn adaptive gradient-based optimization algorithm that updates model parameters using running estimates of first and second moments of gradients.
labelThe desired output values that a model is trained to match for each input example.
mean absolute errorThe average absolute difference between predictions and numeric targets, used as a regression error metric.
batch sizeThe number of sequence examples processed together in one forward or training step.
generationThe process of producing new text token by token from a trained language model, often by feeding previous outputs back as inputs.
cross-validationA validation procedure that partitions data into K folds, trains on K−1 folds, and evaluates on the remaining fold in rotation.
latent spaceAn encoded internal representation in latent space that preserves the structure needed for generation or manipulation.
preprocessingThe transformation of raw data into a form suitable for input to a model.
objective functionA function that compares predictions with targets and returns a scalar indicating how wrong the model is for a given task.
machine learning workflowThe end-to-end sequence of steps for defining, building, evaluating, deploying, and maintaining a machine learning system.
performanceA measurement of business outcome or value used to judge the effect of a deployed model.
IMDb datasetA movie-review dataset used here as the source of training and validation reviews for fine-tuning.
arrayA numerical container whose elements are organized over one or more axes and manipulated by vectorized operations.
normalizationThe process of rescaling or normalizing features so they have more stable numerical properties for learning.
multi-hot encodingA representation scheme where multiple positions are set to 1 and all others to 0 to indicate the presence or absence of several categories or symbols.
categorical encodingA representation scheme where one position is 1 and all others are 0 to encode a discrete category or symbol.
scalabilityThe practice of increasing model size, data, or training compute to improve performance in machine learning.
classification headThe final neural-network layer that maps learned representations to class predictions.
language modelA language model trained on large text corpora that can generate human-sounding responses, mentioned here as an example of a system that may be modular but still lacks recombination ability.
embeddingAn ordered numeric representation used here to encode source tokens and target tokens for attention computations.
validation_data argumentA fit() argument that supplies a separate dataset for computing validation loss and metrics during training.
multilabel classificationA classification task in which each example may belong to multiple classes at the same time, even though there are multiple possible classes overall.
gradient descentA parameter update step computed from gradients to reduce model error.
dataset splitOne of the K disjoint subsets created when data is split for cross-validation.
token idInteger identifiers assigned to tokens so text can be represented as discrete numeric symbols.
training datasetThe subset of data used to fit a model's parameters from examples, here emphasized as being small in size.
neural network layerA neural-network building block that transforms one or more input tensors into one or more output tensors and may carry learned state.
vectorizationThe process of converting data into vector form so it can be used by machine-learning code.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.