Feature engineering turns raw data into representations designed by people, often domain experts, before a learning algorithm uses them. In computer vision, handcrafted features encoded assumptions about useful visual patterns and invariances; a shallow classifier then mapped those features to outputs. section 2.1
Before AlexNet, vision systems commonly combined local descriptors such as SIFT or SURF with global representations such as bag-of-visual-words and spatial pyramids. SIFT was designed for scale and rotation variation; HOG captured edge patterns useful for shape detection. Researchers fed these features to methods such as SVMs and boosted decision trees. section 2.1
When a feature misleads
A HOG-based detector mistakes rippled water beside a goose for a car. The classifier acts on the features it receives; the failure lies in a representation that makes irrelevant edges resemble car-related patterns. A learned CNN feature could handle such a case better. section 2.1
Figure 2.1 This figure shows a goose swimming (top), where an object detector mistakenly labels a small patch of rippled water as a car. Below the image are three visualizations that clarify the source of this error: a close-up of the water patch (left), the corresponding HOG features (middle), and a specialized visualization (right) showing that the rippled water’s features resemble those associated with cars in HOG feature space. Used with permission granted by the lead author (Carl Vondrick).
Compare the water patch with its HOG features and the visualization of its similarity to car-related features. section 2.1
AlexNet marked a shift away from handcrafted image features: in 2012, it achieved a 15% top-five error rate, compared with about 26% for conventional methods. In chemistry, hand-engineered molecular descriptors likewise contrast with message-passing neural networks, which learn directly from molecular data. section 2.0 section 7.3
