Hyperparameter tuning searches for configuration values that improve model performance. Treat it as part of the training pipeline: first identify likely low-hanging experiments, then choose a search strategy that matches dataset size, experiment duration, and available compute. section 10.4
A practical tuning workflow includes these decisions:
- Select parameters that are genuinely changeable for the pipeline; do not make every model function configurable by default. section 10.4
- Use handcrafted experiments when compute is limited or individual runs take weeks; use random search or Bayesian optimization when resources support many runs. section 10.4
- Record parameters, input data, feature engineering, metrics, and model artifacts so experiments can be compared and managed. section 10.6.1
- Protect the test set: repeatedly searching against one validation split can overfit it and produce biased, nonrobust results. section 7.1
Forecasting at Supermegaretail
A retail forecasting pipeline can tune configurations across baseline, classical ML, and deep learning models using grid search or Bayesian optimization. It should judge candidates with MAE, MSE, RMSE, quantile metrics, confidence intervals, and business costs for overstock and out-of-stock situations. section 10.6.1
Listing 10.1 Multilevel hierarchy of subcodings
def train():
...
batch_size = 32
...
learning_rate = 3e-4
...
model.train(data, batch_size, loss_fn)
...
^ example of underconfigured code: things are too rigid
class Config(BaseConfig):
def __init__(self):
self.data_config = DataConfig()
self.model_config = ModelConfig()
self.training_config = TrainingConfig()
self.inference_config = InferenceConfig()
self.environment_config = EnvironmentConfig()
class DataConfig(BaseConfig):
def __init__(self):
self.train_data_config = TrainDataConfig()
self.validation_data_config = ValidationDataConfig()
self.test_data_config = TestDataConfig()
config = Config(
data_config=DataConfig(
train_data_config=TrainDataConfig(
...
),
validation_data_config=ValidationDataConfig(
...
),
test_data_config=TestDataConfig(
...
),
),
...
)
A hierarchical configuration example shows how data, model, training, inference, and environment settings can be separated instead of hard-coding values inside the training function. section 10.4
