Optuna tunes neural-network hyperparameters when gradient descent cannot choose them directly, such as learning rate, momentum, layer count, and hidden-unit counts. It replaces costly grid-search combinations with sequential trials guided by a learned model and an available compute budget. section 5.4 section 5.4.1
A practical Optuna workflow is:
- Define an objective function that accepts a trial, requests parameter guesses with trial.suggest functions, trains or evaluates a model, and returns one numeric score to minimize or maximize. section 5.4.1
- Create a study with the desired direction and call study.optimize with a trial count; Optuna uses each result to improve later guesses. section 5.4.1
- Inspect study.best_params or visualize the search, then retrain a model using the selected hyperparameters and evaluate it on held-out data. section 5.5
A toy minimization problem
For f(x, y) = abs((x − 3)(y + 2)), the objective samples x and y over supplied ranges and returns the function value. After 100 trials, the reported best parameters approach x = 3 and y = −2, illustrating how trial results guide subsequent guesses. section 5.4.1
