Optuna
Optuna is an open-source Python framework for automated hyperparameter optimization through repeated trials that explore parameter configurations.
Inside Deep Learning: Math, Algorithms, Models
A hyperparameter-optimization library that searches parameter spaces by running trials and using a model to guide new suggestions.
- How it is used
- Tunes neural-network hyperparameters and architectures when optimization objectives lack gradients.
- How deep it goes
- Explains objective functions, bounded suggestions, studies, trials, and best-parameter retrieval, but defers Optuna’s technical internals.
- Examples
- Uses a two-variable toy function, trial logs, best-parameter output, contour visualization, and a FashionMNIST neural-network exercise.
- What you can do after
- Apply Optuna by writing an objective, running trials, retrieving parameters, and evaluating a retrained neural network.
Read how Inside Deep Learning: Math, Algorithms, Models explains Optuna →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
Optuna is an open-source hyperparameter optimization framework that adaptively searches parameter configurations through repeated trials.
- How it is used
- Deploys adaptive Bayesian search to refine XGBoost hyperparameters for classification and regression workflows.
- How deep it goes
- Explains studies, objectives, trial suggestions, evaluation choices, storage, and visual diagnostics, but defers algorithmic internals.
- Examples
- Uses XGBoost classification and Tokyo Airbnb regression, including optimization histories, parameter-importance plots, and reported best configurations.
- What you can do after
- Apply it via Python to construct studies, run configurable trials, select best parameters, and inspect optimization results.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains Optuna →
Designing Deep Learning Systems: A guide for software engineers
Optuna is an open-source Python library for automated hyperparameter optimization with trial pruning, parallel execution, and visualization.
- How it is used
- Automates hyperparameter searches for model training on local machines or managed clusters.
- How deep it goes
- Explains objective functions, studies, suggestions, pruning, and parallel setup, but remains an overview rather than search-algorithm internals.
- Examples
- Uses scikit-learn SVR and RandomForest regressors, sampled parameters, dataset splitting, and mean squared error.
- What you can do after
- Apply via Optuna by defining an objective, creating a study, setting a trial quota, and optionally sharing relational storage.
Read how Designing Deep Learning Systems: A guide for software engineers explains Optuna →
Related concepts
- MIPROv2used by · MIPROv2 is a DSPy optimizer that searches for effective instructions and demonstrations for language-model programs using performance feedback and a Bayesian proposal model.
- surrogate modeluses · A surrogate model is an approximate model that estimates the behavior or outputs of a more complex, costly, or otherwise unavailable model, process, or function.
- hyperparameter tuningused by · Hyperparameter tuning is the process of selecting or adjusting configurable model or procedure settings and evaluating them against a chosen objective.
- tree-structured parzen estimatoruses · A tree-structured Parzen estimator is a density-estimation model that organizes conditional distributions in a tree to guide sequential hyperparameter optimization.
- Bayesian optimizationkind of · Bayesian optimization is a sequential method for optimizing an expensive or unknown objective by using a probabilistic model to guide evaluations.
- librarykind of · A software library is a reusable collection of code or components that other programs can incorporate or invoke to provide functionality.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.