A feature is an input that helps a machine-learning model learn a pattern and make a prediction. It may be a familiar value, such as a product’s price, or a derived representation, such as a vector produced by another model. The practical question is not how many inputs you can supply, but which ones capture useful information for the prediction. section 6.2.3 section 11.0 section 11.1.1
Start with a baseline before building an elaborate feature pipeline. For grocery demand forecasting, a simple prediction can reuse an item’s sales from one week earlier. A linear-model baseline can then use past sales at several lags and aggregates over rolling windows. Product brand, shop location, price, and day of the week offer further candidates, provided they are preprocessed for the chosen model. section 8.6.1
When looking for the next useful input, consider three routes:
- Add an available column or a new internal or external data source. This can improve predictions but adds pipeline and maintenance work. section 11.1.2
- Transform an existing value: scale or normalize a numeric feature, create a time lag, or calculate an aggregate over a past window. Whether a transformation helps depends on the model. section 11.1.2
- Combine existing inputs, then test whether the result improves the model enough to justify its cost. section 11.1 section 11.1.2
Judge a candidate on more than an offline performance gain. Check its relationship to the target, the historical data available to compute it, interpretability, computation time, and the reliability of its sources. Many redundant inputs can encourage overfitting; a complex input can also make a production pipeline fragile. Even a statistically significant improvement may not justify a feature that takes days to compute. section 11.1.1 section 11.5.1
Search relevance has features, too
A photo-search baseline can score the distances between a query and an image and its description. Making description and metadata usable as inputs still requires transformation. Later versions might add document click-through rate, user click-history aggregates, or photographer ratings. Keeping the scoring design extensible makes those additions possible without requiring them on day one. section 11.5.2
Handcrafted columns are not the only option. Another model can produce representations of users, products, images, text, or audio for a downstream model. Whether the input is readable or a vector, plan how it will be generated, tested, and maintained before relying on it. section 6.2.3 section 11.1
