liveBook · Manning

Hyperparameter tuning

Also called hyperparameter optimization, hyperparameter search

Hyperparameter tuning is the systematic selection of configurable model or learning-procedure settings by evaluating alternatives against a chosen objective.

How 22 Manning books explain it

Designing Deep Learning Systems: A guide for software engineers

Hyperparameter optimization in which software algorithms select values and manage trials with minimal human intervention.

How it is used
Automates selection of training settings to improve model results within a trial budget.
How deep it goes
Explains trial workflows, search categories, service architecture, and library patterns while deferring algorithm mathematics.
Examples
Demonstrates classifier selection with Hyperopt, regression tuning with Optuna, and ConvNet tuning with Ray Tune.
What you can do after
Apply it via Hyperopt, Optuna, or Ray Tune by defining an objective, search space, algorithm, and trial limit.

Read how Designing Deep Learning Systems: A guide for software engineers explains hyperparameter tuning →

Evolutionary Deep Learning: Genetic algorithms and neural networks

Hyperparameter optimization, or hyperparameter tuning, is the process of selecting configuration values that control a machine-learning model or training procedure to improve a chosen objective.

How it is used
Improves deep-learning models by selecting training and model control variables through manual or automated search.
How deep it goes
Works through runnable random, grid, PSO, evolutionary-strategy, and differential-evolution searches, including Python configuration generators and fitness evaluation.
Examples
Uses a PyTorch network approximating a polynomial function while varying epochs, hidden-layer size, learning rate, and batch size.
What you can do after
Apply it via Python and DEAP to compare search methods, record fitness, and select promising deep-learning configurations.

Read how Evolutionary Deep Learning: Genetic algorithms and neural networks explains hyperparameter tuning →

Build AI Drug Discovery Pipelines

Hyperparameter tuning is the systematic selection of model configuration values that are not learned directly from the training examples.

How it is used
Uses tuning to improve molecular prediction and parameter sweeps to compare active learning configurations.
How deep it goes
Works through pipeline construction, search spaces, cross-validation, and search results for grid and randomized search.
Examples
Uses cardiotoxicity classification with Morgan fingerprints and an SGD classifier, plus neural-network training and validation curves.
What you can do after
Apply it via Scikit-Learn pipelines, GridSearchCV, and RandomizedSearchCV to compare molecular model configurations.

Read how Build AI Drug Discovery Pipelines explains hyperparameter tuning →

Bayesian Optimization in Action

Hyperparameter tuning is the process of selecting machine-learning configuration values that optimize a specified performance objective.

How it is used
Uses model evaluations to choose configurations that improve a specified performance metric under limited trials.
How deep it goes
Defines queries and expensive black-box objectives, then develops Bayesian search while stopping short of a complete tuning workflow.
Examples
Uses neural-network depth, decision-tree settings, SVM parameters, an accuracy table, and simulated SVM accuracy surfaces.
What you can do after
Apply it via Bayesian optimization to prioritize promising model configurations when evaluations are expensive.

Read how Bayesian Optimization in Action explains hyperparameter tuning →

Transformers in Action

Hyperparameter optimization is the systematic search for hyperparameter values that improve a model's training or evaluation performance.

How it is used
Improves large-model training and evaluation by searching settings that affect convergence and performance.
How deep it goes
Works through data preparation, configurable training, search setup, result selection, and retraining, alongside sweep configuration and tracking.
Examples
Uses learning-rate comparisons and a SuperGLUE CommitmentBank classification search, plus a Bayesian sweep configuration.
What you can do after
Apply it via Ray Tune or Weights & Biases by configuring searches, comparing evaluation metrics, and selecting settings.

Read how Transformers in Action explains hyperparameter tuning →

Deep Learning with R, Third Edition

Hyperparameter tuning is the process of selecting model configuration values by evaluating alternative settings during development.

How it is used
Improves model configurations systematically during development before final test evaluation.
How deep it goes
Explains iterative search and demonstrates Bayesian optimization with a model builder, validation split, early stopping, and ranked results.
Examples
Uses an MNIST classifier to compare hidden-layer widths and optimizers.
What you can do after
Apply it via KerasTuner to search candidate configurations and retrieve the best hyperparameters.

Read how Deep Learning with R, Third Edition explains hyperparameter tuning →

Building Specialized AI Systems

The systematic selection of training configuration values to improve model performance.

How it is used
Guides fine-tuning configuration for domain-specialized models while protecting foundational knowledge.
How deep it goes
Explains learning rate, batch size, epochs, gradient accumulation, overfitting, and early stopping without optimization algorithms.
Examples
Uses refining an existing skill as an analogy and task-specific summarization, classification, and generation as model outcomes.
What you can do after
Evaluate/understand configuration trade-offs among adaptation speed, stability, compute resources, and generalization.

Read how Building Specialized AI Systems explains hyperparameter tuning →

Deep Learning with Python, Third Edition

The process of searching over model configuration choices to find settings that improve validation or test performance.

How it is used
Uses tuning to improve generalization during model development and automate configuration search.
How deep it goes
Explains the search loop and gradient-free choices, then walks through tuner configuration, search, and retraining.
Examples
Tunes an MNIST classifier over hidden units and optimizer choices, and suggests Kaggle competitions for practice.
What you can do after
Apply it via KerasTuner to define a search space, run trials, and retrieve configurations for retraining.

Read how Deep Learning with Python, Third Edition explains hyperparameter tuning →

Software Engineering for Data Scientists

Hyperparameter tuning is the process of selecting configuration values for a machine learning algorithm to optimize model performance.

How it is used
Optimizes model configurations during training while supporting reusable model classes, progress monitoring, and parallel execution.
How deep it goes
Works through parameter grids, randomized search, scoring, fitted estimators, predictions, monitoring, and parallel runtime; it does not explain search internals.
Examples
Uses customer churn classification and artists popularity regression with random forest and gradient boosting parameter grids.
What you can do after
Apply it via scikit-learn RandomizedSearchCV to tune models, inspect the best estimator, monitor progress, and configure parallel jobs.

Read how Software Engineering for Data Scientists explains hyperparameter tuning →

Causal Inference for Data Science

Hyperparameter tuning is the process of selecting model-training settings by evaluating candidate settings and choosing the best-performing configuration.

How it is used
Selects machine-learning settings before fitting nuisance models in cross-fitted double machine learning.
How deep it goes
Explains candidate evaluation, fold-specific integration, and DoubleML support while deferring exact R and Python syntax to package documentation.
Examples
Uses k-nearest-neighbor k values, linear-regression regularization, two-fold DML, and a right-heart catheterization treatment-effect exercise.
What you can do after
Operate/configure DoubleML by supplying candidate hyperparameters and enabling fold-specific tuning for an ATE analysis.

Read how Causal Inference for Data Science explains hyperparameter tuning →

Natural Language Processing in Action, Second Edition

Hyperparameter tuning is the systematic process of selecting hyperparameter values to improve a model or processing pipeline.

How it is used
Improves NLP pipelines and neural-network models by selecting configurations that produce better retrieval or predictive performance.
How deep it goes
Defines tuning, demonstrates parameter changes, compares logged experiments, and discusses Bayesian automation without prescribing a complete search algorithm.
Examples
Uses CountVectorizer 2-grams, a command-line CNN search, learning curves, and logged RNN comparisons.
What you can do after
Apply it via recorded experiments and parameter comparisons to improve retrieval, accuracy, training speed, or overfitting behavior.

Read how Natural Language Processing in Action, Second Edition explains hyperparameter tuning →

Machine Learning System Design

Hyperparameter optimization is the process of searching for configuration values that improve a model's performance.

How it is used
Guides training-pipeline experiments toward configurations that improve model performance under practical compute constraints.
How deep it goes
Explains parameter selection, manual and automated search, experiment tracking, and validation risks without deriving search algorithms.
Examples
Uses retail forecasting, grid search, Bayesian optimization, random search, MLflow tracking, and hierarchical configuration files.
What you can do after
Apply it via random search, Bayesian optimization, grid search, or controlled manual experiments while tracking results and protecting evaluation data.

Read how Machine Learning System Design explains hyperparameter tuning →

Transfer Learning for Natural Language Processing

The process of searching for hyperparameter values that improve a model's performance.

How it is used
Uses broader hyperparameter evaluation to improve model performance and make algorithm comparisons more reliable.
How deep it goes
Walks through intuition-led changes and a 27-combination grid search, without exhaustively tuning every possible setting.
Examples
Compares email and IMDB classification while varying token limits and tuning random-forest parameters.
What you can do after
Apply hyperparameter search with GridSearchCV to evaluate a specified parameter grid and identify strong settings.

Read how Transfer Learning for Natural Language Processing explains hyperparameter tuning →

Advanced Algorithms and Data Structures

Parameter fine-tuning is the systematic refinement of tested parameter values around the most promising settings.

How it is used
Uses systematic parameter search to improve genetic-algorithm results.
How deep it goes
Explains a coarse-to-fine protocol and combination costs, but does not develop a formal optimization method.
Examples
Uses mutation-chance ranges and a TSP table comparing crossover and swap-operator chances.
What you can do after
Apply coarse-to-fine search to select promising values for several genetic-algorithm parameters.

Read how Advanced Algorithms and Data Structures explains hyperparameter tuning →

Data Analysis with LLMs

Model tuning is the adjustment of a model's generation settings to improve its behavior on a task.

How it is used
Adjusts generation settings to constrain review-label outputs while improving classification quality and reducing unused token generation.
How deep it goes
Demonstrates token restrictions and a one-token output limit, but notes these are only a subset of available tuning options.
Examples
Compares ten review labels and reports seven correct results after limiting output to one token, up from six.
What you can do after
Apply output constraints through the OpenAI Python client to generate single-token class labels and limit response length.

Read how Data Analysis with LLMs explains hyperparameter tuning →

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

Hyperparameter optimization, or tuning, is the process of searching for hyperparameter settings that improve a model's evaluation objective.

How it is used
Improves model evaluation results by systematically testing hyperparameter settings within a defined search space.
How deep it goes
Explains search-space design, cross-validation, algorithm choice, and a 60-trial optimization, without deriving search algorithms mathematically.
Examples
Uses a Tokyo Airbnb XGBoost regressor, Optuna trials, MAE, booster choices, objectives, and parameter-importance visualizations.
What you can do after
Apply it via Optuna to evaluate candidate XGBoost configurations and retrieve suitable hyperparameters for a regression objective.

Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains hyperparameter tuning →

Distributed Machine Learning Patterns

Hyperparameter tuning is the process of selecting hyperparameter values by evaluating models trained with different configurations.

How it is used
Optimizes TensorFlow model configurations before selecting a final model for evaluation.
How deep it goes
Explains search spaces, Hyperband, validation objectives, early stopping, and final retraining without detailing tuner internals.
Examples
Uses an MNIST dense neural network with tunable layer units and Adam learning rates.
What you can do after
Apply it via Keras Tuner to search configurations, retrieve the best hyperparameters, retrain, and evaluate a model.

Read how Distributed Machine Learning Patterns explains hyperparameter tuning →

Optimization Algorithms: AI techniques for design, planning, and control problems

Parameter tuning is the process of selecting suitable values for an algorithm's configurable parameters before or during execution.

How it is used
Selects or updates algorithm parameters to improve search performance for problem-specific optimization tasks.
How deep it goes
Explains timing, update modes, parameter trade-offs, and sensitivity analysis through tabu search and particle swarm optimization.
Examples
Uses tabu tenure, swarm size, and neighborhood size to illustrate tuning trade-offs.
What you can do after
Apply it via experimentation, sensitivity analysis, or adaptive updates to choose problem-specific algorithm parameters.

Read how Optimization Algorithms: AI techniques for design, planning, and control problems explains hyperparameter tuning →

Inside Deep Learning: Math, Algorithms, Models

The process of searching over user-chosen model settings to find values that improve an objective score.

How it is used
Uses a numeric objective to search model settings for improved accuracy or reduced error.
How deep it goes
Explains Optuna’s sequential Bayesian search and trial interface, but defers the algorithm’s technical details.
Examples
Uses a two-variable toy function and discusses searching regularization penalties for linear models.
What you can do after
Apply it via Optuna by defining an objective, parameter ranges, a study direction, and a trial count.

Read how Inside Deep Learning: Math, Algorithms, Models explains hyperparameter tuning →

LLMs in Production

Hyperparameter tuning is the process of selecting values for training or model-configuration choices that are not learned as model parameters.

How it is used
Guides choices among training settings and generation controls when improving model performance or response behavior.
How deep it goes
Defines temperature through logits and softmax, but treats broad LLM tuning briefly and defers deeper optimization to data and model changes.
Examples
Uses Optuna, the prompt “I am a,” temperature settings, logits, softmax probabilities, and beam search as teaching examples.
What you can do after
Apply it via Optuna for training searches or configure temperature and beam settings through a compatible LLM API endpoint.

Read how LLMs in Production explains hyperparameter tuning →

Machine Learning Platform Engineering

Hyperparameter optimization is the process of selecting hyperparameter values by evaluating configurations within a defined search space against an objective such as a loss function.

How it is used
Deploys automated configuration search during model training to improve target metrics while reducing manual steps.
How deep it goes
Defines search spaces, objectives, automation, and modularity but defers specific search algorithms and implementation details.
Examples
Uses a defined search space, a loss function, tracked checkpoints, and a retraining pipeline with versioned components.
What you can do after
Operate automated searches reproducibly by supplying configurations, data, code, a search space, and an objective.

Read how Machine Learning Platform Engineering explains hyperparameter tuning →

Financial AI in Practice

Hyperparameter tuning is the process of selecting configuration values for a model’s learning procedure to improve its performance.

How it is used
Uses search to select model-learning settings against a task-specific validation metric, such as fraud recall.
How deep it goes
Works through randomized search, fold-level resampling, refitting, and test evaluation; mentions grid and Bayesian search without implementing them.
Examples
Uses a fraud pipeline tuned for recall, fixed XGBoost settings in credit scoring, and conservative random forest settings for market prediction.
What you can do after
Apply scikit-learn RandomizedSearchCV to select and refit a fraud-detection pipeline before evaluating it on unseen test data.

Read how Financial AI in Practice explains hyperparameter tuning →

Also covered in

What to know first

Related concepts

Questions about hyperparameter tuning

What is hyperparameter tuning?

Hyperparameter tuning is the systematic selection of configurable model or learning-procedure settings by evaluating alternatives against a chosen objective.

Which Manning books explain hyperparameter tuning?

Designing Deep Learning Systems: A guide for software engineers, Evolutionary Deep Learning: Genetic algorithms and neural networks, Build AI Drug Discovery Pipelines, Bayesian Optimization in Action, Transformers in Action, Deep Learning with R, Third Edition, Building Specialized AI Systems, Deep Learning with Python, Third Edition and 14 more.

What should I know before learning hyperparameter tuning?

Manning books that teach hyperparameter tuning usually introduce accuracy, objective function, language model and regularization first.

What is hyperparameter tuning also called?

Hyperparameter tuning is also called hyperparameter optimization or hyperparameter search.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.