How would you decide whether two fruits are alike? You could compare their size and how red they are. Each measurement is a feature: a characteristic you use to describe and compare an item. section 12.3
Feature extraction turns those characteristics into numbers. A fruit can become a pair of numbers, while a person's movie preferences can become five category ratings. These lists of numbers are vectors. You can then calculate the distance between vectors: a smaller distance indicates more similar sets of numbers. section 12.3
How many loaves should you bake?
Suppose you know how many loaves a bakery sold on previous days. Describe each day using weather rated from 1 to 5, whether it was a weekend or holiday, and whether a game was on. For today, find the four past days with the closest feature values and average their sales. In this example, the result is 218.75 loaves. This is KNN used for regression: predicting a number from nearby examples. section 12.4
Which features should you pick?
- Choose features related to the prediction. Ratings of cat pictures probably say less about movie taste than movie-category ratings do. section 12.5
- Watch for narrow or biased inputs. Ratings from only one kind of movie may miss other tastes; a user's viewing habits may tell you something their ratings do not. section 12.5
- Be cautious when the future depends on many variables. A stock's movement yesterday, for instance, cannot guarantee its movement today. section 12.9
Features sit between collecting and cleaning data and training a model. After extracting them, you can train on part of the data and check predictions against held-back examples. A distance formula is one way to compare feature vectors; cosine similarity is another, but its use is left for further exploration. section 12.10 section 12.4
