Covariance matrix

As explained in Bayesian Optimization in Action

A covariance matrix is a matrix whose diagonal contains variances and whose off-diagonal entries contain covariances between components of a multivariate random variable.

A covariance matrix summarizes how multiple random variables vary together. In a multivariate Gaussian, its diagonal contains each variable’s variance, while entry i,j contains the covariance between variables Xi and Xj. section 2.2.1

Read the matrix by separating individual uncertainty from relationships:

• Diagonal entries describe the variance of the corresponding variables. section 2.2.1
• Off-diagonal entries describe covariance, indicating how two variables change together; larger positive values represent stronger positive association. section 2.2.1
• In high-dimensional data, nearby variables can have larger covariances while distant variables have covariances near zero. section 2.2.3

Housing prices along a street

Treat house prices as variables in a 20-dimensional Gaussian. Neighboring houses may have covariance around 0.87, whereas houses at opposite ends may have covariance near zero. Observing one house’s price therefore provides more information about nearby prices than distant ones; conditioning updates their means and reduces their uncertainty. section 2.2.3

Figure 2.10 Heat map showing the covariance matrix of a 20-dimensional Gaussian distribution. Neighboring variables are more correlated than those that are far away, as indicated by a darker shade.

A covariance heat map makes the relationship structure visible: darker neighboring entries indicate stronger covariance, while distant pairs approach zero. section 2.2.3

How Bayesian Optimization in Action covers it

How deep it goes

Defines diagonal and off-diagonal meanings, then illustrates distance-dependent covariance and conditioning without deriving matrix computations.

Examples

Uses housing prices, a 20-dimensional street model, covariance heat maps, and conditioned Gaussian visualizations.

What it assumes

Assumes familiarity with normal distributions and random-variable vectors, while introducing multivariate Gaussian notation from those ideas.

What you can do after

Evaluate how observed variables should update predictions and uncertainty for correlated variables.

How it connects

Connects covariance matrices to multivariate Gaussian distributions, correlation, Bayesian conditioning, posterior distributions, and Gaussian processes.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Math and Architectures of Deep LearningA matrix summarizing variances and covariances among dimensions.
• 

  liveBookmanning.com
  Build a Robo-Advisor with Python (From Scratch)A covariance matrix is a square symmetric matrix containing asset variances on its diagonal and pairwise covariances off the diagonal.
• 

  liveBookmanning.com
  Outlier Detection in PythonA covariance matrix is a square matrix containing the variances of variables on its diagonal and their pairwise covariances elsewhere.
• 

  liveBookmanning.com
  Data Without LabelsA covariance matrix is a square matrix whose entries represent pairwise covariances among variables.

Compare how 5 books explain covariance matrix →

Related concepts

• principal component analysisused by · Principal component analysis is a linear dimensionality-reduction technique that transforms data into orthogonal components ordered by the variance they capture.
• normal distributionpart of · A normal distribution is a continuous probability distribution with a symmetric, bell-shaped density determined by its mean and variance.
• Gaussian processused by · A Gaussian process is a probability distribution over functions in which every finite collection of function values has a joint Gaussian distribution.
• tracking errormeasures · Tracking error is the variability or magnitude of an investment's return difference from the benchmark or reference portfolio it is intended to follow.
• risk contributionused by · A risk contribution is the fraction of a portfolio's variance risk attributable to an asset or subportfolio, calculated as its covariance with the full portfolio divided by the portfolio variance.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.