Mean absolute error (MAE) measures the average absolute distance between predictions and targets. An MAE of 0.5 means predictions miss by 0.5 target units on average. section 4.3.3
Use MAE as a regression metric rather than classification accuracy. A model can train with mean squared error while reporting MAE for easier interpretation. section 4.3.3 section 4.3.6
Temperature forecasting baseline
Predict tomorrow’s temperature by repeating the current temperature, then average the absolute errors. This commonsense baseline reaches validation MAE 2.43°C and test MAE 2.62°C, giving a practical benchmark for learned models. section 13.2.2
Listing 13.10 Computing the commonsense baseline MAE
evaluate_naive_method <- function(dataset) {
.[temp_sd = sd, temp_mean = mean] <- normalization_values$`T (degC)`
unnormalize_temperature <- function(x) {
(x * temp_sd) + temp_mean
}
temp_col_idx <- match("T (degC)", input_data_colnames)
reduction <- dataset |>
dataset_unbatch() |>
dataset_map(function(samples, target) {
last_temp_in_input <- samples@r[-1, temp_col_idx] #1
pred <- unnormalize_temperature(last_temp_in_input) #2
abs(pred - target)
}) |>
dataset_reduce(
initial_state = list(total_samples_seen = 0L,
total_abs_error = 0),
reduce_func = function(state, element) {
`add<-` <- `+`
add(state$total_samples_seen) <- 1L
add(state$total_abs_error) <- element
state
}
) |>
lapply(as.numeric) #3
mae <- with(reduction, total_abs_error / total_samples_seen) #4
mae
}
sprintf("Validation MAE: %.2f", evaluate_naive_method(val_dataset))
sprintf("Test MAE: %.2f", evaluate_naive_method(test_dataset))
[1] "Validation MAE: 2.43"
[1] "Test MAE: 2.62"
Computes absolute errors across a dataset, accumulates their total, and divides by the number of samples to evaluate the baseline. section 13.2.2
