Prior knowledge is information or assumptions about an objective function available before all relevant data is observed. In a Gaussian process, it supplies an informed starting point instead of assuming the function is zero everywhere. section 3.0 section 3.2
A GP expresses this knowledge through its mean and covariance functions. The mean describes expected behavior; the covariance describes structure, smoothness, variability, and relationships between inputs. section 3.2 section 3.6
Useful prior knowledge can describe:
- Bounds, such as model accuracy lying between 0 and 1. section 3.2
- Signs and trends, such as housing prices being positive and increasing with desirable property features. section 3.2
- Different sensitivity across dimensions, represented through covariance choices such as separate length scales. section 3.2 section 3.6
Temperature estimates
Before collecting detailed observations, one can reasonably expect a typical January day in Missouri to be relatively cold and a summer day in California to be relatively hot. These rough expectations become initial Bayesian-model guesses. section 3.0
Figure 3.4 Comparison between a prior and a posterior GP. The prior GP contains prior information about the objective function, while the posterior GP combines that information with actual observations.
Compare a prior GP, which contains prior information, with a posterior GP that combines that information with observations. section 3.2
