liveBook · Manning

Feature extraction

As explained in Applied Reinforcement Learning

Feature extraction is the transformation of raw state information into numerical features that represent properties relevant to prediction.

In reinforcement learning, feature extraction turns a raw state into a vector of numbers, each capturing a property relevant to prediction. This compact representation discards detail to make learning tractable. A linear model can weight and sum the features to estimate a state–action value; feature extraction itself does not update the model. section 8.1

Raw states

An image, waveform, patient record, or high-dimensional sensor vector can be a raw state. The relevant features depend on which properties matter for prediction. section 8.1

Figure 8.3 Linear function approximation with temporal-difference learning

See how feature extraction fits into linear function approximation and temporal-difference learning. section 8.1

How Applied Reinforcement Learning covers it

How deep it goes
Defines numerical features and compact abstraction, but does not explain how to engineer particular features.
Examples
Uses images, waveforms, patient records, and sensor vectors as raw-state examples without specifying extracted features.
What you can do after
Understand how raw states become compact numerical inputs to linear value estimation.
How it connects
Connects feature vectors to linear function approximation, value estimation, and temporal-difference learning.

Also explained in

Compare how 2 books explain feature extraction →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.