Hyperparameter

As explained in Machine Learning System Design

A configuration value selected outside model parameter learning that controls training or model behavior.

A hyperparameter is a configuration value chosen outside model-parameter learning. It controls training or model behavior, such as how strongly a loss function emphasizes competing objectives. section 5.3.2 section 9.1.4

Use hyperparameters to make explicit tradeoffs and compare model configurations:

• Set alpha, beta, and gamma to balance classification quality, confidence on difficult examples, and result diversity. section 5.3.2
• Sweep a model-complexity value across a grid, train each configuration to convergence, and compare final loss or metric values. section 9.1.4

Choosing model complexity

For gradient boosting, test tree depths from 2 through 16. Plot each depth against its final metric or loss to identify the useful complexity range and expose overfitting or underfitting concerns. section 9.1.4

Figure 9.7 Finding the optimal model complexity based on learning curves

Use the learning-curve figure to relate candidate hyperparameter values to model complexity and select an effective range. section 9.1.4

How Machine Learning System Design covers it

How deep it goes

Defines hyperparameter grids, trains configurations to convergence, and compares final metrics, without detailing search automation.

Examples

Uses loss weights, gradient-boosting tree depth, regularization strength, feature count, and neural-network layer count.

What you can do after

Evaluate candidate configurations and tune objective balance or model complexity using final loss and metric comparisons.

How it connects

Connects loss functions, metrics, model complexity, learning curves, regularization, gradient boosting, and neural networks.

liveBookmanning.com

Read Machine Learning System Design on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA hyperparameter is a quantity set before training that controls how a machine-learning model is trained or specified.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA hyperparameter is a configuration value selected before or during model training rather than learned directly from the training examples.
• 

  liveBookmanning.com
  Advanced Algorithms and Data StructuresA hyperparameter is a configuration value governing an algorithm's behavior that is set independently of the algorithm's learned or computed results.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA hyperparameter is a user-specified setting that controls a machine-learning algorithm or processing pipeline rather than being learned from its training data.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA hyperparameter is a configuration value selected outside parameter learning that controls a model, training procedure, or optimization process.
• 

  liveBookmanning.com
  Evolutionary Deep Learning: Genetic algorithms and neural networksA hyperparameter is a configuration value governing model structure or training that is set outside ordinary parameter learning.
• 

  liveBookmanning.com
  Data Without LabelsA hyperparameter is a configuration value selected before or during model training that controls an algorithm's behavior or structure.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsUser-chosen settings such as layer counts or learning rate that control model architecture or training behavior rather than being learned.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA user-chosen model or training setting, such as layer count or activation choice, that is set before training and tuned externally rather than learned by backpropagation.
• 

  liveBookmanning.com
  Distributed Machine Learning PatternsA configuration value chosen before or during model training that controls the model structure or training procedure rather than being learned directly from the data.
• 

  liveBookmanning.com
  Transformers in ActionA hyperparameter is a configuration value set before or outside model training that controls architecture or optimization behavior.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchTraining or model settings chosen before fitting, such as layer counts, head counts, embedding size, vocabulary size, block size, and dropout rate.
• 

  liveBookmanning.com
  Experimentation for EngineersA hyperparameter is a setting that controls the behavior or structure of a statistical model rather than representing an observed system input.
• 

  liveBookmanning.com
  Outlier Detection in PythonA hyperparameter is a configuration value set before model training or algorithm execution that controls the model or algorithm's behavior.
• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningA hyperparameter is a configuration value selected before or outside the fitting of a machine-learning model.
• 

  liveBookmanning.com
  AI-Powered SearchA hyperparameter is a configuration value that controls a model or learning procedure and is set rather than learned from individual examples.
• 

  liveBookmanning.com
  Designing Deep Learning Systems: A guide for software engineersA configuration value selected before or outside model fitting that controls a learning algorithm or training process.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthA hyperparameter is a configuration value set outside the ordinary parameter-learning process that controls a model or learning procedure.

Compare how 19 books explain hyperparameter →

Related concepts

• parametercontrasts with · A model parameter is a numerical value, such as a weight or bias, that is learned or set to determine how a model computes outputs.
• regularization parameterincludes · A regularization parameter is a model-fitting setting that controls the influence of a penalty or constraint relative to fit to the data.
• temperatureincludes · A temperature parameter adjusts the concentration of a probability distribution, affecting sampling randomness or the likelihood of accepting less favorable transitions.
• learning rateincludes · A learning rate is a step-size parameter that controls the magnitude of updates made by an iterative learning or optimization process.
• grid searchused by · Grid search is a search method that evaluates configurations formed from specified candidate values, typically every combination in their Cartesian product.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.