Hyperopt

As explained in Designing Deep Learning Systems: A guide for software engineers

An open-source Python library for defining and performing hyperparameter optimization experiments.

Hyperopt is a lightweight Python library for automating hyperparameter optimization in small or early-stage model-training projects. It supports serial and parallel searches, reducing impractical manual tuning across choices such as learning rate, batch size, epochs, and model structure. section 5.0 section 5.4.1

A basic Hyperopt experiment has three implementation steps:

• Define an objective function that receives sampled hyperparameters, trains a model, evaluates it, and returns a numeric score. section 5.4
• Declare a search space, including conditional choices and parameter distributions for each candidate model configuration. section 5.4.1
• Call the optimization routine with the objective, search space, selected algorithm, and trial limit. section 5.4.1

Figure 5.11 An objective function receives the hyperparameters as input and returns a score.

Use the objective-function diagram to trace how sampled hyperparameters enter training and become the score returned to the search algorithm. section 5.4

Classifier search

A practical experiment can compare naive Bayes, SVM, and decision trees. The search space gives SVM-specific values for C and kernel width, while decision trees vary criterion and maximum depth. TPE can optimize the space for ten trials. section 5.4.1

Listing 5.1 Getting started with Hyperopt

# Step 1: define an objective function
def objective(args):
  model = train(args)                                   #1
  return evaluate(model)                                #1
 
# Step 2 define search space for hyperparameters 
space = hp.choice('classifier_type', [                  #2
  {
    'type': 'naive_bayes',
  },
  {
    'type': 'svm',
    'C': hp.lognormal('svm_C', 0, 1),                   #3
    'kernel': hp.choice('svm_kernel', [                 #3
      {'ktype': 'linear'},                              #3
      {'ktype': 'RBF',                                  #3
       'width': hp.lognormal('svm_rbf_width', 0, 1)},   #3
    ]),
  },
  {
    'type': 'dtree',
    'criterion': hp.choice('dtree_criterion', 
      ['gini', 'entropy']),
    'max_depth': hp.choice('dtree_max_depth',
      [None, hp.qlognormal('dtree_max_depth_int', 3, 1, 1)]),
    'min_samples_split': hp.qlognormal(
      'dtree_min_samples_split', 2, 1, 1),
  },
  ])
 
#Step 3 start the hpo process execution 
best = fmin(objective, space, algo=tpe.suggest,
  max_evals=100)                            #4

Use the starter listing as a runnable reference for wrapping training, defining conditional hyperparameter spaces, and invoking Hyperopt. section 5.4.1

For larger runs, distribute Hyperopt workers across machines through a central coordination database or use Spark for parallel execution. section 5.4.1

How Designing Deep Learning Systems: A guide for software engineers covers it

How deep it goes

Defines the objective, search space, algorithm, trials, and parallelization path without detailing algorithm internals.

Examples

Compares naive Bayes, SVM, and decision-tree classifiers using conditional parameter spaces and ten TPE trials.

What it assumes

Assumes training and evaluation code, while teaching the HPO objective and search-space structure explicitly.

What you can do after

Build and run serial or parallel HPO experiments around existing training and evaluation code.

How it connects

Connects hyperparameter optimization with objective functions, search spaces, TPE, distributed workers, and Spark.

liveBookmanning.com

Read Designing Deep Learning Systems: A guide for software engineers on liveBook Get the book at manning.com

Related concepts

• random searchimplements · Random search is an optimization technique that explores a search space by evaluating candidate solutions or configurations selected through randomized choices.
• tree-structured parzen estimatorimplements · A tree-structured Parzen estimator is a density-estimation model that organizes conditional distributions in a tree to guide sequential hyperparameter optimization.
• hyperparameter optimization servicekind of · A software service that coordinates automated searches for effective hyperparameter values by running and comparing multiple training trials.
• Bayesian optimizationuses · Bayesian optimization is a sequential method for optimizing an expensive or unknown objective by using a probabilistic model to guide evaluations.
• Sparkuses · Apache Spark is an open-source distributed computing engine for large-scale data processing and analytics across clusters, supporting batch and streaming workloads.

This explainer was generated by Manning's knowledge graph from the text of Designing Deep Learning Systems: A guide for software engineers. The section links point to the parts of the book it draws on.