Hyperparameter
A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure.
Grokking Machine Learning
A hyperparameter is a quantity set before training that controls how a machine-learning model is trained or specified.
- How it is used
- Controls choices made before training, from learning speed to decision-tree stopping rules.
- How deep it goes
- Defines hyperparameters against learned parameters and lists model-specific choices; defers detailed tuning methods.
- Examples
- Uses regression learning rates, tree depth and sample thresholds, and neural-network architecture and dropout settings.
- What you can do after
- Apply it via grid search to compare model settings while using training, validation, and testing splits.
Read how Grokking Machine Learning explains hyperparameter →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A hyperparameter is a configuration value selected before or during model training rather than learned directly from the training examples.
- How it is used
- Improves tabular-model performance by selecting algorithm settings that fit the data problem and available computational resources.
- How deep it goes
- Defines settings, distinguishes them from learned parameters, and develops search spaces, cross-validation, grid search, and manual tuning trade-offs.
- Examples
- Uses LogisticRegression C, random-forest n_estimators, gradient-boosting learning_rate, and TabNet optimizer, patience, and batch size.
- What you can do after
- Apply it via Scikit-learn search tools or manual settings to compare validated configurations and balance performance against computational cost.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains hyperparameter →
Advanced Algorithms and Data Structures
A hyperparameter is a configuration value governing an algorithm's behavior that is set independently of the algorithm's learned or computed results.
- How it is used
- Guides algorithm behavior when clustering data, optimizing a function, or arranging a graph.
- How deep it goes
- Defines the term through algorithm inputs and works through their effects and selection, without prescribing a universal tuning rule.
- Examples
- Uses cluster-count trials, DBSCAN radius comparisons, gradient-descent step sizes, and attractive-versus-repulsive graph forces.
- What you can do after
- Evaluate/understand how candidate settings change clustering results or optimization behavior, then compare settings using appropriate feedback.
Read how Advanced Algorithms and Data Structures explains hyperparameter →
Grokking Bayes
A hyperparameter is a quantity that controls the form or behavior of a prior or model distribution rather than being directly inferred as an ordinary model parameter.
- How it is used
- Uses hyperparameters to set prior beliefs, update distributions after data, and control exploration in an upper confidence bound rule.
- How deep it goes
- Works through beta updates and prior-choice comparisons, then shows normal-prior updates and a decision-rule setting.
- Examples
- Uses a five-person tea survey, house-price regression, and action utilities to show distinct hyperparameter choices.
- What you can do after
- Apply it via NumPy and SciPy to update beta or normal distribution settings and compare prior and posterior densities.
Natural Language Processing in Action, Second Edition
A hyperparameter is a user-specified setting that controls a machine-learning algorithm or processing pipeline rather than being learned from its training data.
- How it is used
- Uses hyperparameters to configure and improve NLP pipelines and neural-network training.
- How deep it goes
- Explains individual settings and compares experimental results, including task-based evaluation and reproducibility considerations.
- Examples
- Experiments with search n-grams, CNN configurations, and an RNN surname classifier.
- What you can do after
- Apply tuning via CountVectorizer and nlpia2 scripts, comparing settings and observed results.
Read how Natural Language Processing in Action, Second Edition explains hyperparameter →
Deep Learning with R, Third Edition
A hyperparameter is a configuration value selected outside parameter learning that controls a model, training procedure, or optimization process.
- How it is used
- Uses hyperparameters to refine model configuration after establishing a baseline and demonstrating the ability to overfit.
- How deep it goes
- Explains manual and automated search procedures, including validation feedback, search-space definition, and the risk of validation overfitting.
- Examples
- Uses layer sizes, learning rate, and an MNIST classifier with a KerasTuner search over units and optimizers.
- What you can do after
- Apply it via KerasTuner to define and search model configurations using validation performance.
Read how Deep Learning with R, Third Edition explains hyperparameter →
Evolutionary Deep Learning: Genetic algorithms and neural networks
A hyperparameter is a configuration value governing model structure or training that is set outside ordinary parameter learning.
- How it is used
- Guides model and evolutionary-operator configuration to improve deep-learning training and optimization.
- How deep it goes
- Works through editable notebook settings, their interactions and outcomes, then encodes bounded GAN settings as evolutionary genes.
- Examples
- Uses function approximation settings, genetic-algorithm population and mutation rates, and float-gene DCGAN configurations.
- What you can do after
- Apply it via PyTorch, Keras, and evolutionary optimizers by changing, encoding, and comparing model or training controls.
Data Without Labels
A hyperparameter is a configuration value selected before or during model training that controls an algorithm's behavior or structure.
- How it is used
- Controls clustering behavior and neural-network structure through settings supplied for training.
- How deep it goes
- Defines the term, identifies algorithm-specific settings, and outlines repeated training and evaluation without prescribing a universal selection rule.
- Examples
- Uses customer-segmentation iterations and a DBSCAN fit with explicit neighborhood settings.
- What you can do after
- Practice/perform iterative tuning by changing settings, measuring performance, and checking whether resulting clusters meet business needs.
Inside Deep Learning: Math, Algorithms, Models
User-chosen settings such as layer counts or learning rate that control model architecture or training behavior rather than being learned.
- How it is used
- Optimizes architecture and training settings when gradients cannot directly choose values.
- How deep it goes
- Explains definitions, search tradeoffs, validation procedure, and PyTorch implementation, but stops short of exhaustive large-scale tuning.
- Examples
- Uses learning rate, momentum, layer count, hidden-layer width, dynamic nn.Sequential models, and Optuna trials.
- What you can do after
- Apply it via Optuna to search model depth, layer width, and learning rate while preserving an unbiased final validation check.
Read how Inside Deep Learning: Math, Algorithms, Models explains hyperparameter →
Deep Learning with Python, Third Edition
A user-chosen model or training setting, such as layer count or activation choice, that is set before training and tuned externally rather than learned by backpropagation.
- How it is used
- Uses hyperparameters to refine model architecture and training choices for better generalization.
- How deep it goes
- Explains the tuning loop, automated search, and validation-overfitting risk, with a KerasTuner model-building example.
- Examples
- Uses a Keras model with tunable dense-layer units and an RMSprop-or-Adam optimizer choice.
- What you can do after
- Apply tuning via KerasTuner by defining choices, evaluating configurations on validation data, and reserving test data for final evaluation.
Read how Deep Learning with Python, Third Edition explains hyperparameter →
Build AI Drug Discovery Pipelines
A hyperparameter is a configuration value selected before or outside model fitting that controls an algorithm, representation, preprocessing, or model complexity.
- How it is used
- Uses configuration search to balance fitting training molecules against generalizing to unseen production molecules.
- How deep it goes
- Explains candidate values, pipeline integration, cross-validation, and bias-variance effects without developing exhaustive optimization theory.
- Examples
- Demonstrates Morgan fingerprints, SGD classifiers, polynomial features, SVR validation curves, and decision-tree depth and leaf constraints.
- What you can do after
- Apply it via Scikit-Learn pipelines and GridSearchCV or RandomizedSearchCV, then inspect cross-validated scores and selected settings.
Read how Build AI Drug Discovery Pipelines explains hyperparameter →
Distributed Machine Learning Patterns
A configuration value chosen before or during model training that controls the model structure or training procedure rather than being learned directly from the data.
- How it is used
- Supports model architecture experiments and performance-oriented training searches.
- How deep it goes
- Works through defining search spaces, Hyperband search, early stopping, selection, retraining, and evaluation without detailing tuner internals.
- Examples
- Uses MNIST digit classification with tunable dense-layer units and Adam learning rate, plus workflow experiments over architectures and configurations.
- What you can do after
- Apply it via Keras Tuner to search configurations, select the best values, train a model, and evaluate it.
Read how Distributed Machine Learning Patterns explains hyperparameter →
Transformers in Action
A hyperparameter is a configuration value set before or outside model training that controls architecture or optimization behavior.
- How it is used
- Frames hyperparameters as choices governing LLM training and model architecture.
- How deep it goes
- Explains gradient-descent effects and training trade-offs, and identifies tools for organizing hyperparameter searches.
- Examples
- Illustrates learning-rate convergence and configures LoRA rank in an Unsloth example.
- What you can do after
- Evaluate hyperparameter choices by reasoning about convergence, overfitting, architecture, and low-rank updates.
Learn Generative AI with PyTorch
Training or model settings chosen before fitting, such as layer counts, head counts, embedding size, vocabulary size, block size, and dropout rate.
- How it is used
- Uses pretraining and model-training settings to configure architectures, control optimization, and select training duration.
- How deep it goes
- Explains selection through validation and implements settings in GPT and MuseGAN configurations, but does not derive systematic search strategies.
- Examples
- Uses image-classification validation, GPT-2XL configuration, and MuseGAN training-loop settings as concrete examples.
- What you can do after
- Apply it via PyTorch configurations and training loops, then evaluate settings with validation loss and generated output.
Read how Learn Generative AI with PyTorch explains hyperparameter →
Experimentation for Engineers
A hyperparameter is a setting that controls the behavior or structure of a statistical model rather than representing an observed system input.
- How it is used
- Controls Gaussian process regression behavior by setting the smoothness of its surrogate function.
- How deep it goes
- Defines sigma, explains its kernel effect, demonstrates weighted estimates, and defers LOOCV details.
- Examples
- Uses sigma values, two parameter measurements, weighted kernel estimates, and a Python Gaussian process regression implementation.
- What you can do after
- Apply it via Python to adjust Gaussian process smoothness and estimate measurements from nearby observations.
Read how Experimentation for Engineers explains hyperparameter →
Outlier Detection in Python
A hyperparameter is a configuration value set before model training or algorithm execution that controls the model or algorithm's behavior.
- How it is used
- Controls detector behavior and supports systematic comparison of outlier-detection configurations.
- How deep it goes
- Explains parameter effects, sweeps, evaluation, and robust ranges, but stops short of deriving detector-specific optimization theory.
- Examples
- Uses baseball-data DBSCAN and KNN experiments, PyOD detectors, synthetic anomalies, and AUROC-based comparisons.
- What you can do after
- Tune detector configurations, compare parameter ranges, and build ensembles less dependent on a single setting.
Read how Outlier Detection in Python explains hyperparameter →
Ensemble Methods for Machine Learning
A hyperparameter is a configuration value selected before or outside the fitting of a machine-learning model.
- How it is used
- Guides model configuration, complexity control, validation, and overfitting management during ensemble training.
- How deep it goes
- Explains named parameters, their operational effects, cross-validation search, refitting, and early stopping across boosting libraries.
- Examples
- Uses LightGBM, CatBoost, and AdaBoost searches with tree, iteration, learning-rate, sampling, and regularization settings.
- What you can do after
- Apply it via scikit-learn, LightGBM, or CatBoost search utilities to compare configurations and refit a selected model.
Read how Ensemble Methods for Machine Learning explains hyperparameter →
AI-Powered Search
A hyperparameter is a configuration value that controls a model or learning procedure and is set rather than learned from individual examples.
- How it is used
- Configures model training, recommendation representations, and sentence-embedding behavior before learning begins.
- How deep it goes
- Explains definitions, effects, example values, and grid search, but stops short of detailing search algorithms or optimization theory.
- Examples
- Uses Transformer trainer settings, latent-feature dimensions, and RoBERTa sentence-embedding configuration as concrete examples.
- What you can do after
- Apply hyperparameters through Hugging Face TrainingArguments and compare settings with grid search.
Designing Deep Learning Systems: A guide for software engineers
A configuration value selected before or outside model fitting that controls a learning algorithm or training process.
- How it is used
- Controls model-training behavior before execution, including quality, speed, memory use, and learning dynamics.
- How deep it goes
- Defines static training configuration and distinguishes it from learned model parameters through concrete examples, without deriving optimization internals.
- Examples
- Uses learning rate, batch size, epochs, optimizers, ReLU, layer counts, embedding dimensions, and TensorFlow Playground experiments.
- What you can do after
- Apply it via TensorFlow Playground by changing training and architecture settings and observing accuracy, training time, and learning curves.
Read how Designing Deep Learning Systems: A guide for software engineers explains hyperparameter →
Timeless Algorithms, The Foundational Ideas
A hyperparameter is a setting that controls a model or learning procedure and is selected outside ordinary parameter fitting.
- How it is used
- Separates training-learned weights and biases from settings chosen to shape learning and network architecture.
- How deep it goes
- Defines settings, explains effects on capacity and training, and describes validation-based selection without guaranteeing a globally best configuration.
- Examples
- Names depth, width, activation functions, learning rate, batch size, regularization strength, tree count, and decision-tree depth.
- What you can do after
- Evaluate candidate configurations on validation data, then reserve test data for final evaluation.
Read how Timeless Algorithms, The Foundational Ideas explains hyperparameter →
Grokking Machine Learning, Second Edition
A quantity that is set before the training process and controls how a model is trained.
- How it is used
- Controls training choices such as speed, duration, data batching, model structure, and stopping conditions.
- How deep it goes
- Defines the parameter distinction and lists tuning examples, stopping short of detailed search procedures or optimization internals.
- Examples
- Uses perceptron learning rate and epochs, tree depth and sample limits, and neural-network architecture, regularization, and dropout.
- What you can do after
- Evaluate training settings and choose among hyperparameter combinations using performance on a validation set.
Read how Grokking Machine Learning, Second Edition explains hyperparameter →
Fight Fraud with Machine Learning
A hyperparameter is a configuration value selected before or during model training that controls the model or training procedure.
- How it is used
- Guides configuration of machine-learning training for fraud, illicit-transaction, and voice-faking detection models.
- How deep it goes
- Defines hyperparameters, distinguishes them from learned parameters, and demonstrates selection, training, validation, and metric inspection.
- Examples
- Uses tree depth and class weights for classical models, then gives an MLP configuration with epochs, learning rate, Adam, and weight decay.
- What you can do after
- Apply hyperparameters via Torch to train, validate, and evaluate neural fraud-detection models while comparing learning behavior and metrics.
Read how Fight Fraud with Machine Learning explains hyperparameter →
Machine Learning Algorithms in Depth
A hyperparameter is a configuration value set outside the ordinary parameter-learning process that controls a model or learning procedure.
- How it is used
- Guides model selection and training configuration toward validation performance and generalization on unseen data.
- How deep it goes
- Defines configuration choices, derives Bayesian optimization with Gaussian processes, and explains learning-rate dynamics without fully detailing every optimizer.
- Examples
- Uses nearest-neighbor counts, K-means cluster counts, SVM kernels, regularization constants, learning rates, and momentum values.
- What you can do after
- Apply it via grid search, random search, or Bayesian optimization to configure SVMs, random forests, and neural-network optimizers.
Read how Machine Learning Algorithms in Depth explains hyperparameter →
LLM Customization and Fine-Tuning
A configurable setting that controls model training rather than being learned directly from training data.
- How it is used
- Uses hyperparameters to control full-parameter supervised fine-tuning under memory and stability constraints.
- How deep it goes
- Explains individual settings and their tradeoffs, then gives an ordered, one-variable-at-a-time tuning procedure with stopping criteria.
- Examples
- Uses starting settings for 4B–7B models and an illustrative six-run accuracy log that plateaus below target.
- What you can do after
- Diagnose and tune full-parameter SFT runs using loss curves and one-variable comparisons, then consider more training data when gains plateau.
Read how LLM Customization and Fine-Tuning explains hyperparameter →
Machine Learning System Design
A configuration value selected outside model parameter learning that controls training or model behavior.
- How it is used
- Balances competing objectives in a loss function and selects model complexity during experimentation.
- How deep it goes
- Defines hyperparameter grids, trains configurations to convergence, and compares final metrics, without detailing search automation.
- Examples
- Uses loss weights, gradient-boosting tree depth, regularization strength, feature count, and neural-network layer count.
- What you can do after
- Evaluate candidate configurations and tune objective balance or model complexity using final loss and metric comparisons.
Read how Machine Learning System Design explains hyperparameter →
Machines that Think
A hyperparameter is a value selected outside the model's learned parameters to control training or model behavior.
- How it is used
- Controls how training proceeds, with the learning rate setting the size of gradient-descent updates.
- How deep it goes
- Defines hyperparameters through the learning rate and explains how step size affects training, without detailing broader hyperparameter choices.
- Examples
- Uses a “hot and cold” search analogy and a loss-landscape figure to contrast cautious and overly bold steps.
- What you can do after
- Understand how learning-rate choices can make training slow, steady, or unstable.
Also covered in
- Applied Reinforcement LearningIn early access (MEAP)
- Build a Text-to-Image Generator (from Scratch)
- Causal AI
- Transfer Learning for Natural Language Processing
- Bayesian Optimization in Action
- Machine Learning Platform Engineering
- Software Testing with Generative AI
- Statistics Every Programmer Needs
- Context EngineeringIn early access (MEAP)
- Software Engineering for Data ScientistsIn early access (MEAP)
- The Art of AI Product Development
- Grokking AI Algorithms, Second Edition
What to know first
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- overfittingOverfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
- objective functionAn objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
- epochAn epoch is one complete pass through the training data during model training.
Related concepts
- parametercontrasts with · A parameter is a value or named input that specifies or influences the behavior, computation, or output of a function, model, or system.
- regularization parameterincludes · A regularization parameter is a model-fitting setting that controls the influence of a penalty or constraint relative to fit to the data.
- temperatureincludes · A temperature parameter controls the spread of a probability distribution or the likelihood of selecting less-favored alternatives in stochastic sampling or search.
- trainingused by · Training is the process of fitting a machine-learning model to data by adjusting its parameters or learned representations to improve its performance on a task.
- learning rateincludes · A learning rate is a numerical step-size parameter that controls the magnitude of successive updates to model parameters or estimates during iterative learning or optimization.
- hyperparameter tuningused by · Hyperparameter tuning is the systematic selection of configurable model or learning-procedure settings by evaluating alternatives against a chosen objective.
Questions about hyperparameter
What is hyperparameter?
A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure.
Which Manning books explain hyperparameter?
Grokking Machine Learning, Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI, Advanced Algorithms and Data Structures, Grokking Bayes, Natural Language Processing in Action, Second Edition, Deep Learning with R, Third Edition, Evolutionary Deep Learning: Genetic algorithms and neural networks, Data Without Labels and 18 more.
What should I know before learning hyperparameter?
Manning books that teach hyperparameter usually introduce accuracy, overfitting, objective function and epoch first.
What is the difference between hyperparameter and parameter?
A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure. A parameter is a value or named input that specifies or influences the behavior, computation, or output of a function, model, or system.
What kinds of hyperparameter are there?
Kinds of hyperparameter include regularization parameter, temperature and learning rate.
What is hyperparameter also called?
Hyperparameter is also called hyper-parameter.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

























