A missing value is an unavailable entry in a dataset. It is a data-quality problem because many machine-learning and deep-learning algorithms cannot consume missing inputs directly, yet the absence itself can sometimes carry useful predictive information. section 2.2.4 section 9.6
Start by counting missing entries by column and asking why they are absent. A missing value may result from unmatched rows after a database join, or from the data-generating process itself. For example, absent review-derived values can mean that an accommodation has no reviews, rather than that an ordinary value was unrecorded. section 2.2.4 section 7.1.5
(data.isna()
.sum(axis=0)
)
Use pandas `isna()` and column-wise sums to identify which features contain NaN values and how many cases are affected before choosing a remedy. section 2.4.2
Choose treatment according to the feature’s meaning and modeling role:
- Impute a numeric value, commonly the mean, median, or mode; simple univariate imputation uses the same column, while multivariate imputation uses other columns. section 2.2.4 section 2.4.2
- Add a binary missingness indicator when absence may reveal a predictive pattern. section 2.2.4 section 2.4.2
- Use a meaningful boundary or placeholder value when it represents the situation faithfully, such as `-1` for unavailable review measures whose valid values are otherwise positive. section 7.1.5
- Use domain-specific defaults for feature values, such as a median number of bathrooms, zero car parks, or an explicit unknown category. section 9.2.1
- Remove records when the missing field is the prediction target, as with listings lacking a price. section 9.2.2
Cleaning real-estate listing fields
A property dataset can replace missing Bathrooms with the column median, missing Car Parks with zero, and missing categorical fields with distinct unknown labels. The choices reflect domain knowledge: a residence rarely has no bathroom, whereas omitted parking may reasonably indicate no parking space. section 9.2.1
Algorithm choice does not remove the need to investigate meaning. XGBoost and LightGBM can handle missing information, but their default behavior may resemble an average case; explicit encoding is preferable when absence represents a distinct, legitimate condition. Deep-learning workflows require values to be filled or records eliminated before training. section 2.2.4 section 7.1.5 section 9.6
