chapter seven

7 Number go up (or down)! Correlation and linear regression

 

In this chapter

  • The Pearson correlation and how it serves as a hypothesis test for a linear relationship between two variables
  • How to predict the values of correlated variables using linear regression
  • Metrics and assumptions for validating correlation and linear regression models

Linear regression is a type of statistical and machine learning model that fits a linear function between independent (input) and dependent (output) variables given some data. This way, a line fitted to the data can be used to make predictions on data not seen before, assuming there is indeed a linear relationship between the variables. So far, we have focused on only one variable at a time. But it can be helpful to predict or understand hypothesized relationships between multiple variables, such as how much growth a plant will experience based on the number of hours of sunlight it receives. Sometimes these relationships resemble a straight-line pattern, which can make predictions more straightforward. Linear relationships may sound elementary and basic, but they are a foundational part of even the most advanced models in statistics and machine learning. Therefore, they’re a great building block to master.

Linear regression has many strengths, making it a workhorse for many statistical and machine learning models:

Correlation

The Pearson correlation

Hypothesis testing the correlation coefficient

Assumptions of Pearson correlation

Correlation is not causation

Linear regression

Simple linear regression

Interpolation vs. extrapolation

Residuals and sum of squares

Overfitting and bias/variance tradeoff

Evaluating a simple linear regression

A real-world example

Summary