liveBook · Manning

Linear function approximation

As explained in Applied Reinforcement Learning

A function-approximation technique that estimates values as weighted sums of state or state-action features.

When a value table is impractical because the state space is too large, continuous, noisy, or high-dimensional, linear function approximation represents a state or state–action pair with features and estimates its value as a weighted sum. Learning adjusts the weights to fit evidence. This enables generalization beyond visited states, but introduces representation error and training instability. section 8.1

Calibrating a house-price estimate

For house prices, a linear model uses size to predict price: the intercept sets a baseline, and the weight on size controls how much the estimate changes with each unit of size. Adjusting the weight to reduce prediction errors illustrates parameter fitting; in reinforcement learning, targets depend on current predictions. section 8.1

Figure 8.2 Learning as parameter adjustment: fitting a linear model by turning the θ knobs

See parameter adjustment as fitting a line to observed points. section 8.1

How Applied Reinforcement Learning covers it

How deep it goes
Defines weighted-feature estimates and parameter fitting, then contrasts generalization with representation error and training instability.
Examples
Uses house size and price to illustrate fitting a line by adjusting its weight.
What you can do after
Understand how weighted features estimate values and why approximation trades exact lookup for generalization.
How it connects
Connects feature vectors and parameter learning to value functions, Q-values, and tabular methods.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.