A Gaussian process is a probability distribution over functions: every finite collection of function values has a joint Gaussian distribution. Intuitively, it extends Bayesian linear regression beyond a single straight-line relationship. section 3.16 section 3.18
Linear regression assumes that labels follow a linear relationship with features. More complex data may call for higher-degree polynomials; pushing that flexibility to the extreme gives a Gaussian process, interpretable as an infinite-degree polynomial model. section 3.16
The key progression is:
- Linear regression models a first-degree polynomial relationship. section 3.16
- Higher-degree polynomial models represent increasingly complex trends. section 3.16
- Gaussian processes generalize Bayesian linear regression to infinite-degree polynomials while retaining distributional uncertainty. section 3.18



