Feature engineering uses knowledge of a problem and a learning algorithm to make its inputs easier to learn from. The transformations are hardcoded, not learned by the model. It can improve generalization when raw data presents a needlessly difficult problem. section 5.4.2 section 5.4.1
Reading a clock
Suppose the input is an image of a clock and the target is the time. Learning directly from pixels calls for a computationally demanding ConvNet. A short Python script could instead locate the tips of the hands and pass their (x, y) coordinates to a simpler learner. Convert those coordinates to angles around the clock’s center, and the representation becomes simpler still: rounding and a lookup can recover the approximate time without machine learning. section 5.4.2
For a practical first pass:
- Filter out uninformative inputs with feature selection. section 6.2.3
- Use problem knowledge to construct features likely to help, then try to beat a simple baseline. section 6.2.3
- Look for relevant prior work rather than assuming every useful representation must be invented anew. section 6.2.3
Deep learning reduces the need to hand-design representations by learning features during training, but it does not make thoughtful input design irrelevant. Keep manual feature engineering distinct from feature extraction with a pretrained model: that technique reuses features the model learned earlier, then trains a new classifier on them. section 1.7 section 8.3.1
