Missing at random (MAR) describes missingness that can be explained by observed variables, but not by the missing values themselves after conditioning on observed data. For example, another recorded feature may predict whether a value is absent. This differs from missing completely at random (MCAR), where missingness is unrelated to observed and unobserved variables, and missing not at random (MNAR), where the missing value itself helps explain its absence. section 6.1.1
Use the missingness mechanism to guide preprocessing:
- Investigate why values are missing before selecting an algorithm or imputation method. section 6.1.1 section 7.1.5
- For MAR data, consider multivariate imputation: estimate each missing feature from relationships among the other observed features. section 6.1.1
- Treat built-in missing-value handling in gradient-boosted decision trees cautiously for MAR data; imputation or missingness indicators may perform better. section 6.1.2
Accommodation reviews
An accommodation dataset contains missing values for days_since_last_review, reviews_per_month, and score. The missing counts match accommodations with no reviews, so missingness reflects a legitimate business condition rather than MAR. Replacing these values with -1 preserves that signal for models instead of allowing built-in GBDT handling to treat them as an average case. section 7.1.5
Listing 6.2 Multivariate imputation
from sklearn.experimental import (
enable_iterative_imputer
) #1
from sklearn.impute import SimpleImputer, IterativeImputer
from sklearn.ensemble import RandomForestRegressor
Xm = data[continuous].copy() #2
missing_percentage = 0.05
np.random.seed(0)
mask = np.random.rand(*Xm.shape) < missing_percentage #3
Xm[mask] = np.nan
simple_imputer = SimpleImputer()
Xm_si = simple_imputer.fit_transform(Xm) #4
rf = RandomForestRegressor(random_state=0, n_jobs=-1) #5
multivariate_imputer = IterativeImputer(
estimator=rf,
max_iter=1,
tol=0.01
) #6
Xm_mi = multivariate_imputer.fit_transform(Xm) #7
mae = pd.DataFrame(
{
"simple": np.mean(
np.abs(data[continuous] - Xm_si), axis=0
),
"multivariate": np.mean(
np.abs(data[continuous] - Xm_mi), axis=0
)
},
index = continuous
) #8
print(mae)
Compare simple imputation with scikit-learn iterative multivariate imputation using a random forest estimator, then evaluate reconstruction error. section 6.1.1
