Data transformation changes raw feature values into representations that help a model learn. In classical machine learning, transformations extract relevant information, fit model assumptions, address nonlinear relationships, and incorporate domain knowledge. section 1.3
For gradient-boosted decision trees, transform known nonlinearities toward linear forms, bin or discretize values, and precompute interactions. These changes can reduce the number of binary splits and place splits more effectively. section 6.1.4
Use transformations to:
- Extract informative signals while reducing irrelevant or noisy information. section 1.3
- Convert data into a representation compatible with model assumptions and limitations. section 1.3
- Make nonlinear feature-target relationships easier for linear models to approximate. section 1.3
- Encode domain knowledge when business, social, economic, or physical context reveals useful features. section 1.3
Diagnosing transformations with partial dependence
Fit a model, then inspect partial dependence plots to see how predicted targets change as one feature varies. Use the resulting view to identify linearity, nonlinearity, or potential interactions before defining transformations. section 6.1.4
Figure 6.1 Comparison of predictions between a neural network and a bagged trees ensemble for a random dataset with a noisy sine function
Compare how a neural network and a bagged tree ensemble approximate a noisy nonlinear function, clarifying why model choice affects transformation needs. section 6.1.4
