liveBook · Manning

Pearson correlation coefficient

As explained in Grokking Statistics

Pearson correlation is the standardized covariance-based measure of linear association between two numerical variables.

The Pearson correlation coefficient, written r, scores how closely two numerical variables follow a straight-line pattern. Think of temperature and sports drink sales: when warmer days generally mean higher sales, r is positive; when one rises as the other falls, r is negative. section 7.1 section 7.2

Its value runs from -1 to 1. Values near 1 indicate a tight upward linear relationship, values near -1 a tight downward one, and values near 0 little linear relationship. The coefficient is standardized covariance: how the variables move together relative to their variation. section 7.2

Figure 7.3 Different Pearson correlation values with different datasets. Note how a strong positive correlation (close to 1) or negative correlation (close to -1) has data that more closely resembles a line.

Compare the point clouds: stronger positive or negative correlations look more like a line, while weaker ones are more scattered. section 7.2

Temperature and sales

For 10 days of temperature and sports-drink sales, a manual calculation and SciPy's pearsonr both give r = 0.9889: a very strong positive linear association. That describes the observed pattern; it does not establish that temperature causes every change in sales. section 7.1 section 7.2 section 7.13

Use Pearson correlation carefully:

  • Plot the data first. A coefficient can hide outliers, skew, or a pattern that is not adequately linear. section 7.12 section 7.13
  • Check whether the data meet linear-analysis conditions, including homoscedasticity and no influential outliers. section 7.13
  • Pair r with a p-value when testing whether a linear relationship is statistically significant, but do not treat either metric as automatic proof. section 7.0 section 7.13
  • Do not turn association into causation or extrapolate the apparent trend beyond the observed data range. section 7.1 section 7.13

How Grokking Statistics covers it

How deep it goes
Defines and interprets r, shows its covariance formula and SciPy calculation, while deferring formal proofs.
Examples
Uses temperature versus sports-drink sales and airplane height versus bird-strike incidents after binning and log transformation.
What it assumes
Introduces the coefficient from scatterplot patterns and explains sigma, means, covariance, and standard deviations in practical terms.
What you can do after
Apply it via SciPy to quantify and cautiously interpret a linear association with r and a p-value.
How it connects
Connects covariance, standard deviation, scatterplots, p-values, linear regression, homoscedasticity, and outliers.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Statistics. The section links point to the parts of the book it draws on.