Suppose a model predicts how long the nearest taxi will take to reach a customer. A more complex model is not automatically the best next step: a prediction based on the customer's district might already be hard to beat. Feature engineering asks a related question: how should the available data be transformed into inputs that help a model make useful predictions? section 8.3 section 6.2.3
Feature engineering is the iterative work of creating and testing new features or transforming existing ones to improve model performance. It draws on domain knowledge, creativity, and data engineering, and may require new pipelines. Because that work consumes substantial modeling resources, plan the order of experiments rather than generating features indiscriminately. Good inputs can matter more than a more sophisticated model. section 11.1 section 11.0
Treat feature choices as system-design decisions:
- Start with a baseline. Try simple predictions and features before investing in more elaborate models or inputs; retain complexity only when it helps with the task. section 8.3
- Specify computation. Record how each feature is calculated and how any model-generated representation is produced, alongside the data sources, filtering rules, and labels. section 6.7
- Check production availability. Data used during training must also be obtainable when the model serves predictions, within the system's service requirements. section 11.1.1
- Test, monitor, and revise. Fragile inputs or external dependencies can make predictions unstable. Track feature versions and dependencies as business goals and data distributions change. section 11.1.1
A district-level arrival estimate
For the taxi arrival problem, one baseline always predicted five minutes; another returned a fixed estimate for a particular borough. A related approach takes the median arrival time by location. Such grouped predictions test whether location already captures useful signal before a team commits to a more complex model. The stakes are practical: an overestimate may drive a customer away, while an underestimate breaks the promised wait time. section 8.3
Features need not be manually readable columns. Another model may generate vector representations of users, goods, images, text, audio, or video for a downstream model. End-to-end deep learning can replace extensive manual feature construction in some domains, but input design still matters; even a prompt can supply context that directs a model toward relevant information. section 6.2.3 section 11.1
Evaluate a feature set for more than predictive lift. Feature importance can identify influential inputs and support interpretation; selection can improve accuracy, interpretability, and training or serving speed. A feature store can centralize calculation, storage, testing, documentation, and monitoring, especially across multiple tabular sources, but is not a natural fit for pure deep learning problems. section 11.6
Be particularly cautious when a feature comes from another model. Training-time information that will not exist at inference can cause leakage and poor performance on new data. External forecasts may lack the historical versions needed for training, while updated feature extractors can change embeddings unexpectedly. Validate these dependencies, version them, and monitor their outputs. section 11.1.3
