In reinforcement learning, feature extraction turns a raw state into a vector of numbers, each capturing a property relevant to prediction. This compact representation discards detail to make learning tractable. A linear model can weight and sum the features to estimate a state–action value; feature extraction itself does not update the model. section 8.1
Raw states
An image, waveform, patient record, or high-dimensional sensor vector can be a raw state. The relevant features depend on which properties matter for prediction. section 8.1
Figure 8.3 Linear function approximation with temporal-difference learning
See how feature extraction fits into linear function approximation and temporal-difference learning. section 8.1

