liveBook · Manning

Hyperparameter

Also called hyper-parameter

A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure.

How 26 Manning books explain it

Grokking Machine Learning

A hyperparameter is a quantity set before training that controls how a machine-learning model is trained or specified.

How it is used
Controls choices made before training, from learning speed to decision-tree stopping rules.
How deep it goes
Defines hyperparameters against learned parameters and lists model-specific choices; defers detailed tuning methods.
Examples
Uses regression learning rates, tree depth and sample thresholds, and neural-network architecture and dropout settings.
What you can do after
Apply it via grid search to compare model settings while using training, validation, and testing splits.

Read how Grokking Machine Learning explains hyperparameter →

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

A hyperparameter is a configuration value selected before or during model training rather than learned directly from the training examples.

How it is used
Improves tabular-model performance by selecting algorithm settings that fit the data problem and available computational resources.
How deep it goes
Defines settings, distinguishes them from learned parameters, and develops search spaces, cross-validation, grid search, and manual tuning trade-offs.
Examples
Uses LogisticRegression C, random-forest n_estimators, gradient-boosting learning_rate, and TabNet optimizer, patience, and batch size.
What you can do after
Apply it via Scikit-learn search tools or manual settings to compare validated configurations and balance performance against computational cost.

Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains hyperparameter →

Advanced Algorithms and Data Structures

A hyperparameter is a configuration value governing an algorithm's behavior that is set independently of the algorithm's learned or computed results.

How it is used
Guides algorithm behavior when clustering data, optimizing a function, or arranging a graph.
How deep it goes
Defines the term through algorithm inputs and works through their effects and selection, without prescribing a universal tuning rule.
Examples
Uses cluster-count trials, DBSCAN radius comparisons, gradient-descent step sizes, and attractive-versus-repulsive graph forces.
What you can do after
Evaluate/understand how candidate settings change clustering results or optimization behavior, then compare settings using appropriate feedback.

Read how Advanced Algorithms and Data Structures explains hyperparameter →

Grokking Bayes

A hyperparameter is a quantity that controls the form or behavior of a prior or model distribution rather than being directly inferred as an ordinary model parameter.

How it is used
Uses hyperparameters to set prior beliefs, update distributions after data, and control exploration in an upper confidence bound rule.
How deep it goes
Works through beta updates and prior-choice comparisons, then shows normal-prior updates and a decision-rule setting.
Examples
Uses a five-person tea survey, house-price regression, and action utilities to show distinct hyperparameter choices.
What you can do after
Apply it via NumPy and SciPy to update beta or normal distribution settings and compare prior and posterior densities.

Read how Grokking Bayes explains hyperparameter →

Natural Language Processing in Action, Second Edition

A hyperparameter is a user-specified setting that controls a machine-learning algorithm or processing pipeline rather than being learned from its training data.

How it is used
Uses hyperparameters to configure and improve NLP pipelines and neural-network training.
How deep it goes
Explains individual settings and compares experimental results, including task-based evaluation and reproducibility considerations.
Examples
Experiments with search n-grams, CNN configurations, and an RNN surname classifier.
What you can do after
Apply tuning via CountVectorizer and nlpia2 scripts, comparing settings and observed results.

Read how Natural Language Processing in Action, Second Edition explains hyperparameter →

Deep Learning with R, Third Edition

A hyperparameter is a configuration value selected outside parameter learning that controls a model, training procedure, or optimization process.

How it is used
Uses hyperparameters to refine model configuration after establishing a baseline and demonstrating the ability to overfit.
How deep it goes
Explains manual and automated search procedures, including validation feedback, search-space definition, and the risk of validation overfitting.
Examples
Uses layer sizes, learning rate, and an MNIST classifier with a KerasTuner search over units and optimizers.
What you can do after
Apply it via KerasTuner to define and search model configurations using validation performance.

Read how Deep Learning with R, Third Edition explains hyperparameter →

Evolutionary Deep Learning: Genetic algorithms and neural networks

A hyperparameter is a configuration value governing model structure or training that is set outside ordinary parameter learning.

How it is used
Guides model and evolutionary-operator configuration to improve deep-learning training and optimization.
How deep it goes
Works through editable notebook settings, their interactions and outcomes, then encodes bounded GAN settings as evolutionary genes.
Examples
Uses function approximation settings, genetic-algorithm population and mutation rates, and float-gene DCGAN configurations.
What you can do after
Apply it via PyTorch, Keras, and evolutionary optimizers by changing, encoding, and comparing model or training controls.

Read how Evolutionary Deep Learning: Genetic algorithms and neural networks explains hyperparameter →

Data Without Labels

A hyperparameter is a configuration value selected before or during model training that controls an algorithm's behavior or structure.

How it is used
Controls clustering behavior and neural-network structure through settings supplied for training.
How deep it goes
Defines the term, identifies algorithm-specific settings, and outlines repeated training and evaluation without prescribing a universal selection rule.
Examples
Uses customer-segmentation iterations and a DBSCAN fit with explicit neighborhood settings.
What you can do after
Practice/perform iterative tuning by changing settings, measuring performance, and checking whether resulting clusters meet business needs.

Read how Data Without Labels explains hyperparameter →

Inside Deep Learning: Math, Algorithms, Models

User-chosen settings such as layer counts or learning rate that control model architecture or training behavior rather than being learned.

How it is used
Optimizes architecture and training settings when gradients cannot directly choose values.
How deep it goes
Explains definitions, search tradeoffs, validation procedure, and PyTorch implementation, but stops short of exhaustive large-scale tuning.
Examples
Uses learning rate, momentum, layer count, hidden-layer width, dynamic nn.Sequential models, and Optuna trials.
What you can do after
Apply it via Optuna to search model depth, layer width, and learning rate while preserving an unbiased final validation check.

Read how Inside Deep Learning: Math, Algorithms, Models explains hyperparameter →

Deep Learning with Python, Third Edition

A user-chosen model or training setting, such as layer count or activation choice, that is set before training and tuned externally rather than learned by backpropagation.

How it is used
Uses hyperparameters to refine model architecture and training choices for better generalization.
How deep it goes
Explains the tuning loop, automated search, and validation-overfitting risk, with a KerasTuner model-building example.
Examples
Uses a Keras model with tunable dense-layer units and an RMSprop-or-Adam optimizer choice.
What you can do after
Apply tuning via KerasTuner by defining choices, evaluating configurations on validation data, and reserving test data for final evaluation.

Read how Deep Learning with Python, Third Edition explains hyperparameter →

Build AI Drug Discovery Pipelines

A hyperparameter is a configuration value selected before or outside model fitting that controls an algorithm, representation, preprocessing, or model complexity.

How it is used
Uses configuration search to balance fitting training molecules against generalizing to unseen production molecules.
How deep it goes
Explains candidate values, pipeline integration, cross-validation, and bias-variance effects without developing exhaustive optimization theory.
Examples
Demonstrates Morgan fingerprints, SGD classifiers, polynomial features, SVR validation curves, and decision-tree depth and leaf constraints.
What you can do after
Apply it via Scikit-Learn pipelines and GridSearchCV or RandomizedSearchCV, then inspect cross-validated scores and selected settings.

Read how Build AI Drug Discovery Pipelines explains hyperparameter →

Distributed Machine Learning Patterns

A configuration value chosen before or during model training that controls the model structure or training procedure rather than being learned directly from the data.

How it is used
Supports model architecture experiments and performance-oriented training searches.
How deep it goes
Works through defining search spaces, Hyperband search, early stopping, selection, retraining, and evaluation without detailing tuner internals.
Examples
Uses MNIST digit classification with tunable dense-layer units and Adam learning rate, plus workflow experiments over architectures and configurations.
What you can do after
Apply it via Keras Tuner to search configurations, select the best values, train a model, and evaluate it.

Read how Distributed Machine Learning Patterns explains hyperparameter →

Transformers in Action

A hyperparameter is a configuration value set before or outside model training that controls architecture or optimization behavior.

How it is used
Frames hyperparameters as choices governing LLM training and model architecture.
How deep it goes
Explains gradient-descent effects and training trade-offs, and identifies tools for organizing hyperparameter searches.
Examples
Illustrates learning-rate convergence and configures LoRA rank in an Unsloth example.
What you can do after
Evaluate hyperparameter choices by reasoning about convergence, overfitting, architecture, and low-rank updates.

Read how Transformers in Action explains hyperparameter →

Learn Generative AI with PyTorch

Training or model settings chosen before fitting, such as layer counts, head counts, embedding size, vocabulary size, block size, and dropout rate.

How it is used
Uses pretraining and model-training settings to configure architectures, control optimization, and select training duration.
How deep it goes
Explains selection through validation and implements settings in GPT and MuseGAN configurations, but does not derive systematic search strategies.
Examples
Uses image-classification validation, GPT-2XL configuration, and MuseGAN training-loop settings as concrete examples.
What you can do after
Apply it via PyTorch configurations and training loops, then evaluate settings with validation loss and generated output.

Read how Learn Generative AI with PyTorch explains hyperparameter →

Experimentation for Engineers

A hyperparameter is a setting that controls the behavior or structure of a statistical model rather than representing an observed system input.

How it is used
Controls Gaussian process regression behavior by setting the smoothness of its surrogate function.
How deep it goes
Defines sigma, explains its kernel effect, demonstrates weighted estimates, and defers LOOCV details.
Examples
Uses sigma values, two parameter measurements, weighted kernel estimates, and a Python Gaussian process regression implementation.
What you can do after
Apply it via Python to adjust Gaussian process smoothness and estimate measurements from nearby observations.

Read how Experimentation for Engineers explains hyperparameter →

Outlier Detection in Python

A hyperparameter is a configuration value set before model training or algorithm execution that controls the model or algorithm's behavior.

How it is used
Controls detector behavior and supports systematic comparison of outlier-detection configurations.
How deep it goes
Explains parameter effects, sweeps, evaluation, and robust ranges, but stops short of deriving detector-specific optimization theory.
Examples
Uses baseball-data DBSCAN and KNN experiments, PyOD detectors, synthetic anomalies, and AUROC-based comparisons.
What you can do after
Tune detector configurations, compare parameter ranges, and build ensembles less dependent on a single setting.

Read how Outlier Detection in Python explains hyperparameter →

Ensemble Methods for Machine Learning

A hyperparameter is a configuration value selected before or outside the fitting of a machine-learning model.

How it is used
Guides model configuration, complexity control, validation, and overfitting management during ensemble training.
How deep it goes
Explains named parameters, their operational effects, cross-validation search, refitting, and early stopping across boosting libraries.
Examples
Uses LightGBM, CatBoost, and AdaBoost searches with tree, iteration, learning-rate, sampling, and regularization settings.
What you can do after
Apply it via scikit-learn, LightGBM, or CatBoost search utilities to compare configurations and refit a selected model.

Read how Ensemble Methods for Machine Learning explains hyperparameter →

AI-Powered Search

A hyperparameter is a configuration value that controls a model or learning procedure and is set rather than learned from individual examples.

How it is used
Configures model training, recommendation representations, and sentence-embedding behavior before learning begins.
How deep it goes
Explains definitions, effects, example values, and grid search, but stops short of detailing search algorithms or optimization theory.
Examples
Uses Transformer trainer settings, latent-feature dimensions, and RoBERTa sentence-embedding configuration as concrete examples.
What you can do after
Apply hyperparameters through Hugging Face TrainingArguments and compare settings with grid search.

Read how AI-Powered Search explains hyperparameter →

Designing Deep Learning Systems: A guide for software engineers

A configuration value selected before or outside model fitting that controls a learning algorithm or training process.

How it is used
Controls model-training behavior before execution, including quality, speed, memory use, and learning dynamics.
How deep it goes
Defines static training configuration and distinguishes it from learned model parameters through concrete examples, without deriving optimization internals.
Examples
Uses learning rate, batch size, epochs, optimizers, ReLU, layer counts, embedding dimensions, and TensorFlow Playground experiments.
What you can do after
Apply it via TensorFlow Playground by changing training and architecture settings and observing accuracy, training time, and learning curves.

Read how Designing Deep Learning Systems: A guide for software engineers explains hyperparameter →

Timeless Algorithms, The Foundational Ideas

A hyperparameter is a setting that controls a model or learning procedure and is selected outside ordinary parameter fitting.

How it is used
Separates training-learned weights and biases from settings chosen to shape learning and network architecture.
How deep it goes
Defines settings, explains effects on capacity and training, and describes validation-based selection without guaranteeing a globally best configuration.
Examples
Names depth, width, activation functions, learning rate, batch size, regularization strength, tree count, and decision-tree depth.
What you can do after
Evaluate candidate configurations on validation data, then reserve test data for final evaluation.

Read how Timeless Algorithms, The Foundational Ideas explains hyperparameter →

Grokking Machine Learning, Second Edition

A quantity that is set before the training process and controls how a model is trained.

How it is used
Controls training choices such as speed, duration, data batching, model structure, and stopping conditions.
How deep it goes
Defines the parameter distinction and lists tuning examples, stopping short of detailed search procedures or optimization internals.
Examples
Uses perceptron learning rate and epochs, tree depth and sample limits, and neural-network architecture, regularization, and dropout.
What you can do after
Evaluate training settings and choose among hyperparameter combinations using performance on a validation set.

Read how Grokking Machine Learning, Second Edition explains hyperparameter →

Fight Fraud with Machine Learning

A hyperparameter is a configuration value selected before or during model training that controls the model or training procedure.

How it is used
Guides configuration of machine-learning training for fraud, illicit-transaction, and voice-faking detection models.
How deep it goes
Defines hyperparameters, distinguishes them from learned parameters, and demonstrates selection, training, validation, and metric inspection.
Examples
Uses tree depth and class weights for classical models, then gives an MLP configuration with epochs, learning rate, Adam, and weight decay.
What you can do after
Apply hyperparameters via Torch to train, validate, and evaluate neural fraud-detection models while comparing learning behavior and metrics.

Read how Fight Fraud with Machine Learning explains hyperparameter →

Machine Learning Algorithms in Depth

A hyperparameter is a configuration value set outside the ordinary parameter-learning process that controls a model or learning procedure.

How it is used
Guides model selection and training configuration toward validation performance and generalization on unseen data.
How deep it goes
Defines configuration choices, derives Bayesian optimization with Gaussian processes, and explains learning-rate dynamics without fully detailing every optimizer.
Examples
Uses nearest-neighbor counts, K-means cluster counts, SVM kernels, regularization constants, learning rates, and momentum values.
What you can do after
Apply it via grid search, random search, or Bayesian optimization to configure SVMs, random forests, and neural-network optimizers.

Read how Machine Learning Algorithms in Depth explains hyperparameter →

LLM Customization and Fine-Tuning

A configurable setting that controls model training rather than being learned directly from training data.

How it is used
Uses hyperparameters to control full-parameter supervised fine-tuning under memory and stability constraints.
How deep it goes
Explains individual settings and their tradeoffs, then gives an ordered, one-variable-at-a-time tuning procedure with stopping criteria.
Examples
Uses starting settings for 4B–7B models and an illustrative six-run accuracy log that plateaus below target.
What you can do after
Diagnose and tune full-parameter SFT runs using loss curves and one-variable comparisons, then consider more training data when gains plateau.

Read how LLM Customization and Fine-Tuning explains hyperparameter →

Machine Learning System Design

A configuration value selected outside model parameter learning that controls training or model behavior.

How it is used
Balances competing objectives in a loss function and selects model complexity during experimentation.
How deep it goes
Defines hyperparameter grids, trains configurations to convergence, and compares final metrics, without detailing search automation.
Examples
Uses loss weights, gradient-boosting tree depth, regularization strength, feature count, and neural-network layer count.
What you can do after
Evaluate candidate configurations and tune objective balance or model complexity using final loss and metric comparisons.

Read how Machine Learning System Design explains hyperparameter →

Machines that Think

A hyperparameter is a value selected outside the model's learned parameters to control training or model behavior.

How it is used
Controls how training proceeds, with the learning rate setting the size of gradient-descent updates.
How deep it goes
Defines hyperparameters through the learning rate and explains how step size affects training, without detailing broader hyperparameter choices.
Examples
Uses a “hot and cold” search analogy and a loss-landscape figure to contrast cautious and overly bold steps.
What you can do after
Understand how learning-rate choices can make training slow, steady, or unstable.

Read how Machines that Think explains hyperparameter →

Also covered in

What to know first

Related concepts

Questions about hyperparameter

What is hyperparameter?

A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure.

Which Manning books explain hyperparameter?

Grokking Machine Learning, Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI, Advanced Algorithms and Data Structures, Grokking Bayes, Natural Language Processing in Action, Second Edition, Deep Learning with R, Third Edition, Evolutionary Deep Learning: Genetic algorithms and neural networks, Data Without Labels and 18 more.

What should I know before learning hyperparameter?

Manning books that teach hyperparameter usually introduce accuracy, overfitting, objective function and epoch first.

What is the difference between hyperparameter and parameter?

A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure. A parameter is a value or named input that specifies or influences the behavior, computation, or output of a function, model, or system.

What kinds of hyperparameter are there?

Kinds of hyperparameter include regularization parameter, temperature and learning rate.

What is hyperparameter also called?

Hyperparameter is also called hyper-parameter.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.