A new feature is useful only if it adds information the model needs. Redundancy occurs when multiple features provide overlapping information, as with many generated, correlated features. Adding more inputs can improve predictive power, but too many redundant or irrelevant ones can lead to overfitting and worse performance. section 11.1.1
When deciding whether to keep a feature:
- Evaluate its contribution to the project goal rather than counting it as valuable simply because it is available. section 11.1.1
- Prefer a small set of strong, diverse features over a larger set of correlated ones. section 11.1.1
- Weigh any gain against interpretability and the cost of developing and maintaining the feature. section 11.1.1
Feature redundancy is different from duplicated records. Duplicate records can change the data distribution for part of a dataset and affect downstream models; they call for data monitoring rather than a decision about whether overlapping features are worth keeping. section 14.3.6 section 11.1.1
