Hyperparameter
A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure.
Grokking Machine Learning
A hyperparameter is a quantity set before training that controls how a machine-learning model is trained or specified.
- How it is used
- Controls choices made before training, from learning speed to decision-tree stopping rules.
- How deep it goes
- Defines hyperparameters against learned parameters and lists model-specific choices; defers detailed tuning methods.
- Examples
- Uses regression learning rates, tree depth and sample thresholds, and neural-network architecture and dropout settings.
- What you can do after
- Apply it via grid search to compare model settings while using training, validation, and testing splits.
Read how Grokking Machine Learning explains hyperparameter →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A hyperparameter is a configuration value selected before or during model training rather than learned directly from the training examples.
- How it is used
- Improves tabular-model performance by selecting algorithm settings that fit the data problem and available computational resources.
- How deep it goes
- Defines settings, distinguishes them from learned parameters, and develops search spaces, cross-validation, grid search, and manual tuning trade-offs.
- Examples
- Uses LogisticRegression C, random-forest n_estimators, gradient-boosting learning_rate, and TabNet optimizer, patience, and batch size.
- What you can do after
- Apply it via Scikit-learn search tools or manual settings to compare validated configurations and balance performance against computational cost.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains hyperparameter →
Advanced Algorithms and Data Structures
A hyperparameter is a configuration value governing an algorithm's behavior that is set independently of the algorithm's learned or computed results.
- How it is used
- Guides algorithm behavior when clustering data, optimizing a function, or arranging a graph.
- How deep it goes
- Defines the term through algorithm inputs and works through their effects and selection, without prescribing a universal tuning rule.
- Examples
- Uses cluster-count trials, DBSCAN radius comparisons, gradient-descent step sizes, and attractive-versus-repulsive graph forces.
- What you can do after
- Evaluate/understand how candidate settings change clustering results or optimization behavior, then compare settings using appropriate feedback.
Read how Advanced Algorithms and Data Structures explains hyperparameter →
Natural Language Processing in Action, Second Edition
A hyperparameter is a user-specified setting that controls a machine-learning algorithm or processing pipeline rather than being learned from its training data.
- How it is used
- Uses adjustable settings to improve NLP search, topic models, and neural-network training for a chosen task.
- How deep it goes
- Defines the term and works through concrete setting changes and performance comparisons, without prescribing a universal optimum.
- Examples
- Compares CountVectorizer n-grams, CNN configurations, and interactive RNN training for surname classification.
- What you can do after
- Apply it via CountVectorizer or PyTorch by changing settings and checking task-relevant results.
Read how Natural Language Processing in Action, Second Edition explains hyperparameter →
Deep Learning with R, Third Edition
A hyperparameter is a configuration value selected outside parameter learning that controls a model, training procedure, or optimization process.
- How it is used
- Uses hyperparameters to refine a working model’s configuration against validation performance before final test evaluation.
- How deep it goes
- Defines hyperparameters against learned weights and develops a train–validate–repeat search procedure, without prescribing formal rules for choosing values.
- Examples
- Demonstrates an MNIST KerasTuner search over dense-layer units and optimizer choice, with validation accuracy as the objective.
- What you can do after
- Apply it via KerasTuner to compare model configurations on validation data and reserve test data for final evaluation.
Read how Deep Learning with R, Third Edition explains hyperparameter →
Evolutionary Deep Learning: Genetic algorithms and neural networks
A hyperparameter is a configuration value governing model structure or training that is set outside ordinary parameter learning.
- How it is used
- Guides model and evolutionary-operator configuration to improve deep-learning training and optimization.
- How deep it goes
- Works through editable notebook settings, their interactions and outcomes, then encodes bounded GAN settings as evolutionary genes.
- Examples
- Uses function approximation settings, genetic-algorithm population and mutation rates, and float-gene DCGAN configurations.
- What you can do after
- Apply it via PyTorch, Keras, and evolutionary optimizers by changing, encoding, and comparing model or training controls.
Data Without Labels
A hyperparameter is a configuration value selected before or during model training that controls an algorithm's behavior or structure.
- How it is used
- Controls clustering behavior and neural-network structure through settings supplied for training.
- How deep it goes
- Defines the term, identifies algorithm-specific settings, and outlines repeated training and evaluation without prescribing a universal selection rule.
- Examples
- Uses customer-segmentation iterations and a DBSCAN fit with explicit neighborhood settings.
- What you can do after
- Practice/perform iterative tuning by changing settings, measuring performance, and checking whether resulting clusters meet business needs.
Inside Deep Learning: Math, Algorithms, Models
User-chosen settings such as layer counts or learning rate that control model architecture or training behavior rather than being learned.
- How it is used
- Optimizes architecture and training settings when gradients cannot directly choose values.
- How deep it goes
- Explains definitions, search tradeoffs, validation procedure, and PyTorch implementation, but stops short of exhaustive large-scale tuning.
- Examples
- Uses learning rate, momentum, layer count, hidden-layer width, dynamic nn.Sequential models, and Optuna trials.
- What you can do after
- Apply it via Optuna to search model depth, layer width, and learning rate while preserving an unbiased final validation check.
Read how Inside Deep Learning: Math, Algorithms, Models explains hyperparameter →
Deep Learning with Python, Third Edition
A user-chosen model or training setting, such as layer count or activation choice, that is set before training and tuned externally rather than learned by backpropagation.
- How it is used
- Uses hyperparameters to refine model architecture and training settings after a first model demonstrates statistical power.
- How deep it goes
- Defines hyperparameters against backpropagation-trained parameters and works through a KerasTuner search without deriving its search algorithm.
- Examples
- Uses Dense-layer unit counts, optimizer choices, and a KerasTuner search with validation accuracy as its objective.
- What you can do after
- Apply it via KerasTuner to compare configurations on validation data, then evaluate the selected configuration on a held-out test set.
Read how Deep Learning with Python, Third Edition explains hyperparameter →
Distributed Machine Learning Patterns
A configuration value chosen before or during model training that controls the model structure or training procedure rather than being learned directly from the data.
- How it is used
- Supports model architecture experiments and performance-oriented training searches.
- How deep it goes
- Works through defining search spaces, Hyperband search, early stopping, selection, retraining, and evaluation without detailing tuner internals.
- Examples
- Uses MNIST digit classification with tunable dense-layer units and Adam learning rate, plus workflow experiments over architectures and configurations.
- What you can do after
- Apply it via Keras Tuner to search configurations, select the best values, train a model, and evaluate it.
Read how Distributed Machine Learning Patterns explains hyperparameter →
Transformers in Action
A hyperparameter is a configuration value set before or outside model training that controls architecture or optimization behavior.
- How it is used
- Controls learning dynamics, architecture choices, and fine-tuning configurations through values set before training.
- How deep it goes
- Defines the distinction from learned parameters and explains several effects on gradient descent, then identifies tools for searching settings.
- Examples
- Uses learning rate, batch size, and epochs, plus an Unsloth configuration that sets LoRA rank to 32.
- What you can do after
- Operate/configure it by selecting training or LoRA settings and organizing hyperparameter searches with Ray or Weights & Biases.
Learn Generative AI with PyTorch
Training or model settings chosen before fitting, such as layer counts, head counts, embedding size, vocabulary size, block size, and dropout rate.
- How it is used
- Uses pretraining and model-training settings to configure architectures, control optimization, and select training duration.
- How deep it goes
- Explains selection through validation and implements settings in GPT and MuseGAN configurations, but does not derive systematic search strategies.
- Examples
- Uses image-classification validation, GPT-2XL configuration, and MuseGAN training-loop settings as concrete examples.
- What you can do after
- Apply it via PyTorch configurations and training loops, then evaluate settings with validation loss and generated output.
Read how Learn Generative AI with PyTorch explains hyperparameter →
Experimentation for Engineers
A hyperparameter is a setting that controls the behavior or structure of a statistical model rather than representing an observed system input.
- How it is used
- Controls Gaussian process regression behavior by setting the smoothness of its surrogate function.
- How deep it goes
- Defines sigma, explains its kernel effect, demonstrates weighted estimates, and defers LOOCV details.
- Examples
- Uses sigma values, two parameter measurements, weighted kernel estimates, and a Python Gaussian process regression implementation.
- What you can do after
- Apply it via Python to adjust Gaussian process smoothness and estimate measurements from nearby observations.
Read how Experimentation for Engineers explains hyperparameter →
Outlier Detection in Python
A hyperparameter is a configuration value set before model training or algorithm execution that controls the model or algorithm's behavior.
- How it is used
- Controls detector behavior and supports systematic comparison of outlier-detection configurations.
- How deep it goes
- Explains parameter effects, sweeps, evaluation, and robust ranges, but stops short of deriving detector-specific optimization theory.
- Examples
- Uses baseball-data DBSCAN and KNN experiments, PyOD detectors, synthetic anomalies, and AUROC-based comparisons.
- What you can do after
- Tune detector configurations, compare parameter ranges, and build ensembles less dependent on a single setting.
Read how Outlier Detection in Python explains hyperparameter →
Ensemble Methods for Machine Learning
A hyperparameter is a configuration value selected before or outside the fitting of a machine-learning model.
- How it is used
- Guides model configuration, complexity control, validation, and overfitting management during ensemble training.
- How deep it goes
- Explains named parameters, their operational effects, cross-validation search, refitting, and early stopping across boosting libraries.
- Examples
- Uses LightGBM, CatBoost, and AdaBoost searches with tree, iteration, learning-rate, sampling, and regularization settings.
- What you can do after
- Apply it via scikit-learn, LightGBM, or CatBoost search utilities to compare configurations and refit a selected model.
Read how Ensemble Methods for Machine Learning explains hyperparameter →
AI-Powered Search
A hyperparameter is a configuration value that controls a model or learning procedure and is set rather than learned from individual examples.
- How it is used
- Configures model training, recommendation representations, and sentence-embedding behavior before learning begins.
- How deep it goes
- Explains definitions, effects, example values, and grid search, but stops short of detailing search algorithms or optimization theory.
- Examples
- Uses Transformer trainer settings, latent-feature dimensions, and RoBERTa sentence-embedding configuration as concrete examples.
- What you can do after
- Apply hyperparameters through Hugging Face TrainingArguments and compare settings with grid search.
Designing Deep Learning Systems: A guide for software engineers
A configuration value selected before or outside model fitting that controls a learning algorithm or training process.
- How it is used
- Controls model-training behavior before execution, including quality, speed, memory use, and learning dynamics.
- How deep it goes
- Defines static training configuration and distinguishes it from learned model parameters through concrete examples, without deriving optimization internals.
- Examples
- Uses learning rate, batch size, epochs, optimizers, ReLU, layer counts, embedding dimensions, and TensorFlow Playground experiments.
- What you can do after
- Apply it via TensorFlow Playground by changing training and architecture settings and observing accuracy, training time, and learning curves.
Read how Designing Deep Learning Systems: A guide for software engineers explains hyperparameter →
Machine Learning Algorithms in Depth
A hyperparameter is a configuration value set outside the ordinary parameter-learning process that controls a model or learning procedure.
- How it is used
- Guides model selection and training configuration toward validation performance and generalization on unseen data.
- How deep it goes
- Defines configuration choices, derives Bayesian optimization with Gaussian processes, and explains learning-rate dynamics without fully detailing every optimizer.
- Examples
- Uses nearest-neighbor counts, K-means cluster counts, SVM kernels, regularization constants, learning rates, and momentum values.
- What you can do after
- Apply it via grid search, random search, or Bayesian optimization to configure SVMs, random forests, and neural-network optimizers.
Read how Machine Learning Algorithms in Depth explains hyperparameter →
Machine Learning System Design
A configuration value selected outside model parameter learning that controls training or model behavior.
- How it is used
- Balances competing objectives in a loss function and selects model complexity during experimentation.
- How deep it goes
- Defines hyperparameter grids, trains configurations to convergence, and compares final metrics, without detailing search automation.
- Examples
- Uses loss weights, gradient-boosting tree depth, regularization strength, feature count, and neural-network layer count.
- What you can do after
- Evaluate candidate configurations and tune objective balance or model complexity using final loss and metric comparisons.
Read how Machine Learning System Design explains hyperparameter →
Related concepts
- parametercontrasts with · A model parameter is a numerical value, such as a weight or bias, that is learned or set to determine how a model computes outputs.
- regularization parameterincludes · A regularization parameter is a model-fitting setting that controls the influence of a penalty or constraint relative to fit to the data.
- temperatureincludes · A temperature parameter adjusts the concentration of a probability distribution, affecting sampling randomness or the likelihood of accepting less favorable transitions.
- learning rateincludes · A learning rate is a step-size parameter that controls the magnitude of updates made by an iterative learning or optimization process.
- grid searchused by · Grid search is a search method that evaluates configurations formed from specified candidate values, typically every combination in their Cartesian product.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.