Imagine preparing a dataset before teaching a model. Feature selection means choosing useful input columns and removing unnecessary or potentially harmful ones before training. section 13.3 section 13.7
Selection belongs to feature engineering and preprocessing. It follows decisions about numerical and categorical data, including one-hot encoding or binning, and leaves a cleaner set of model inputs. section 13.3
A practical selection pass can look like this:
- Inspect the columns and decide which represent useful information for the prediction task. section 13.3
- Do not treat a model's current feature usage as a complete selection method: greedy decision trees can miss combinations of individually weak features. section 9.5
- For a more deliberate method, consider L1 regularization when selecting features. section 9.5
Titanic preprocessing
After cleaning the passenger dataset, categorical columns such as Sex and Embarked are converted into numerical columns. The original columns can then be removed, while Name, Ticket, and PassengerId are dropped as unused features. section 13.3
