liveBook · Manning

Linear regression

As explained in Timeless Algorithms, The Foundational Ideas

Linear regression is a statistical model that represents a continuous response as a linear function of predictors and model parameters.

Linear regression models a continuous response as a linear function of one or more explanatory variables. It can describe how the response changes with its predictors and predict responses for future observations. section 3.4.1

yi=β0+β1xi+εi

For one explanatory variable, the response yᵢ is expressed using an intercept β₀, slope β₁, predictor xᵢ, and unexplained component εᵢ. section 3.4.1

The slope describes the expected response change for a one-unit predictor increase; the intercept gives the expected response when the predictor is zero. section 3.4.1

y^i=β^0+β^1xi

The fitted model uses estimated intercept and slope values to produce a predicted response for each observation. section 3.4.1

ε^i=yi-y^i

A residual is the difference between an observed response and its fitted prediction; residual size indicates their disagreement. section 3.4.1

Ordinary least squares chooses coefficients to minimize the sum of squared residuals. The aim is to capture the overall relationship, not to pass through every observation. section 3.4.1

With normally distributed errors, this geometric optimization and probabilistic estimation yield the same solution. The connection illustrates how maximum likelihood can unify estimation methods and links regression’s foundation to learning in neural networks. section 3.4.4 section 3.4.4

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines the model, interprets coefficients and residuals, and explains squared-residual fitting and its likelihood connection.
What you can do after
Understand how to interpret regression coefficients, assess residual disagreement, and relate squared-error fitting to likelihood estimation.
How it connects
Connects ordinary least squares to maximum likelihood and relates neural-network learning to the same estimation foundation.

Also explained in

Compare how 28 books explain linear regression →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.