liveBook · Manning

Hyperband

As explained in Distributed Machine Learning Patterns

A hyperparameter-optimization algorithm that allocates computational resources adaptively among configurations using successive halving.

Hyperband tunes model hyperparameters by allocating resources adaptively and stopping weak configurations early, helping searches converge faster on high-performing models. section 8.1.1

A practical Hyperband workflow uses:

  • A model-building function exposing tunable values such as dense-layer units and Adam learning rate. section 8.1.1
  • Validation accuracy as the search objective, with a configured maximum epoch budget and reduction factor. section 8.1.1
  • An EarlyStopping callback monitoring validation loss to terminate training when improvement stalls. section 8.1.1
  • A final training pass that rebuilds the model from the best hyperparameters before test evaluation. section 8.1.1

Keras Tuner configuration

Configure kt.Hyperband with objective='val_accuracy', max_epochs=10, factor=3, and a persistent project directory; run tuner.search with validation_split=0.2 and early stopping. section 8.1.1

Listing 8.1 Creating a Conda environment
> conda create --name dist-ml python=3.9 -y

Use the surrounding setup listings to reproduce the Python environment for the TensorFlow and Keras Tuner workflow. section 8.1.1

How Distributed Machine Learning Patterns covers it

How deep it goes
Explains configuration, search, early stopping, best-parameter retrieval, and final training, but omits algorithmic internals.
Examples
Demonstrates tuning dense-layer units and Adam learning rate for a TensorFlow model, then retraining and evaluating the selected model.
What it assumes
Assumes a TensorFlow model-building function and validation data while demonstrating tunable parameters and tuner setup directly.
What you can do after
Apply it via Keras Tuner to search hyperparameters, recover the best configuration, retrain the model, and evaluate test performance.
How it connects
Connects hyperparameter optimization with Keras Tuner, validation accuracy, early stopping, random search, and Bayesian optimization.
Cover of Distributed Machine Learning Patterns
liveBookmanning.com
Read Distributed Machine Learning Patterns on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Distributed Machine Learning Patterns. The section links point to the parts of the book it draws on.