Linear regression
Linear regression is a statistical method that models a numeric response as a linear function of one or more predictor variables with fitted parameters.
Causal Inference for Data Science
Linear regression is a regression method that estimates a linear relationship between an outcome and one or more predictors.
- How it is used
- Estimates direct or total causal effects by selecting regression covariates from a causal graph.
- How deep it goes
- Works through covariate specifications, do-operator effects, and instrumental-variable coefficient ratios; flags RDD limitations without detailing them.
- Examples
- Uses treatment, recovery, side effects, synthetic confounding data, and R and Python regression or IV code.
- What you can do after
- Evaluate/understand which regression specification targets a direct or total effect and when a naive coefficient is biased.
Read how Causal Inference for Data Science explains linear regression →
Grokking Machine Learning
A regression technique that models a numerical target as a linear function of input features.
- How it is used
- Predicts numerical labels from house features within supervised learning, rather than assigning categories.
- How deep it goes
- Works from a plotted line through iterative parameter updates and Python implementation to real-data modeling, favoring geometry over formal derivation.
- Examples
- Uses room-count house prices, plotted fitting lines and a Hyderabad housing dataset with an RMSE report.
- What you can do after
- Build it from scratch with Python updates or apply it via Turi Create to predict housing prices and inspect RMSE.
Read how Grokking Machine Learning explains linear regression →
Effective Data Analysis
Linearity is the property that a modeled relationship can be represented by additive changes proportional to predictor changes.
- How it is used
- Uses linear regression to model continuous outcomes for prediction and to examine relationships relevant to stakeholder decisions.
- How deep it goes
- Explains equations, assumptions, coefficients, fit metrics, residual diagnostics, and model iteration, with an ordinary least squares implementation.
- Examples
- Uses house prices, weather and rat sightings, and production costs to illustrate modeling and stakeholder tasks.
- What you can do after
- Applies linear regression via statsmodels and evaluates coefficients, fit, and residuals while communicating predictive limitations.
Read how Effective Data Analysis explains linear regression →
Statistics Every Programmer Needs
Linear regression is a statistical modeling technique that represents the relationship between a response variable and one or more predictors with a linear function.
- How it is used
- Models associations and predicts numeric outcomes from known predictors for forecasting and practical decisions.
- How deep it goes
- Works through equations, OLS fitting, output interpretation, and diagnostic tests without pursuing an abstract proof of coefficient estimation.
- Examples
- Fits Stage 2 ultramarathon split times from Stage 1 times and inspects a residuals plot.
- What you can do after
- Apply it via statsmodels OLS to fit a numeric-response model, interpret its output, and test its assumptions.
Read how Statistics Every Programmer Needs explains linear regression →
Math and Architectures of Deep Learning
A function that preserves addition and scalar multiplication between vector spaces, as in matrix multiplication.
- How it is used
- Models quantitative estimation tasks by mapping input features to a numeric prediction.
- How deep it goes
- Derives the weighted-sum-plus-bias model, squared-error objective, and gradient updates, while deferring nonlinear-model details.
- Examples
- Uses noisy y = 1.5x + 2.73 data and weekly mobile-phone demand from sales, discounts, and holiday timing.
- What you can do after
- Build it from scratch with PyTorch by computing predictions, mean squared error, gradients, and weight-and-bias updates.
Read how Math and Architectures of Deep Learning explains linear regression →
Build AI Drug Discovery Pipelines
Linear regression is a regression method that models a linear relationship between input features and a target variable.
- How it is used
- Uses linear regression to predict molecular solubility and support compound prioritization.
- How deep it goes
- Explains weighted predictions, residuals, MSE, closed-form solutions, and gradient descent variants.
- Examples
- Fits solubility from logP and compares descriptor subsets for a small QSAR dataset.
- What you can do after
- Apply it via Scikit-Learn to fit and evaluate a solubility predictor.
Read how Build AI Drug Discovery Pipelines explains linear regression →
Grokking Machine Learning, Second Edition
A function whose output is an affine linear combination of its input variables.
- How it is used
- Predicts numerical labels from labeled features for tasks such as housing prices, spending, and viewing time.
- How deep it goes
- Builds intuition through line fitting, then details iterative parameter updates and a scikit-learn housing workflow.
- Examples
- Uses room counts and house prices, Hyderabad housing data, and video or music viewing-time recommendations.
- What you can do after
- Build it from scratch by iteratively updating slope and intercept, or apply it with scikit-learn to predict housing prices.
Read how Grokking Machine Learning, Second Edition explains linear regression →
Experimentation for Engineers
A regression method that models a response as a linear function of explanatory variables.
- How it is used
- Fits surrogate business-metric models for response-surface optimization and per-action reward models for contextual decisions.
- How deep it goes
- Works through normal-equation and pseudoinverse implementations, quadratic feature construction, interpolation grids, and two-parameter cross terms.
- Examples
- Uses markout profit versus threshold and order size, plus viewing-time prediction from logged user contexts for posts.
- What you can do after
- Apply it via NumPy to fit quadratic surrogate functions, predict unmeasured outcomes, and search for estimated optimal settings.
Read how Experimentation for Engineers explains linear regression →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A statistical model that predicts a numeric response as a linear combination of input variables.
- How it is used
- Establishes an interpretable baseline for tabular numeric prediction before testing more complex models.
- How deep it goes
- Explains numeric encoding, coefficients, residual fitting, preprocessing checks, cross-validation, and diagnostics without lengthy mathematical derivations.
- Examples
- Uses Airbnb listing-price prediction, five-fold RMSE validation, transformed features, diagnostic plots, and a linear-regression-versus-XGBoost comparison.
- What you can do after
- Apply it via scikit-learn to build, validate, inspect, and compare a numeric tabular prediction baseline.
Ensemble Methods for Machine Learning
Linear regression is a model that predicts a target as a linear combination of input variables.
- How it is used
- Predicts continuous labels from weighted input features.
- How deep it goes
- Defines the squared-loss objective and compares OLS, Ridge, LASSO, and Elastic Net regularization.
- Examples
- Uses noisy synthetic data, train-test splitting, fitted coefficients, and MSE or mean absolute error comparisons.
- What you can do after
- Apply linear regression through scikit-learn and evaluate predictions with MSE and mean absolute error.
Read how Ensemble Methods for Machine Learning explains linear regression →
Deep Learning with Python, Third Edition
A transformation that preserves linear structure and can be represented by matrix multiplication.
- How it is used
- Uses affine predictions and squared-error training to illustrate a simple model, alongside continuous-value regression workflows.
- How deep it goes
- Works through an affine classifier’s forward pass and gradient updates, then develops regression using neural networks rather than a standalone linear-regression example.
- Examples
- Uses synthetic two-dimensional class data for the affine model and house-price prediction for scalar regression.
- What you can do after
- Apply affine modeling and regression via TensorFlow and Keras, choosing appropriate output layers, losses, and evaluation metrics.
Read how Deep Learning with Python, Third Edition explains linear regression →
Starting Data Analytics with Generative AI and Python
Linear regression is a statistical modeling technique that estimates a linear relationship between a dependent variable and one or more independent variables.
- How it is used
- Supports business decisions by examining relationships between product attributes and shipping costs.
- How deep it goes
- Works through data preparation, single- and multiple-variable fitting, confidence-interval plots, and model interpretation.
- Examples
- Models freight value using product weight and volume, separately and together.
- What you can do after
- Apply it via statsmodels to fit shipping-cost models and inspect their reported results.
Read how Starting Data Analytics with Generative AI and Python explains linear regression →
Deep Learning with R, Third Edition
A linear transform is a mapping that preserves vector addition and scalar multiplication and can be represented by matrix multiplication.
- How it is used
- Connects affine predictions to continuous-value regression and contrasts them with classification.
- How deep it goes
- Explains affine computation and regression losses, but works through a linear classifier and a nonlinear neural regression model rather than linear regression itself.
- Examples
- Uses a two-feature classifier and a house-price model with a linear output to illustrate related computations.
- What you can do after
- Evaluate and understand how an affine prediction relates to a regression output and its loss.
Read how Deep Learning with R, Third Edition explains linear regression →
Statistics Slam Dunk
A multiple linear regression model represents a target variable as a function of two or more predictor variables, typically using a linear form.
- How it is used
- Models continuous outcomes from multiple predictors to identify and quantify hustle statistics associated with regular-season wins.
- How deep it goes
- Walks through preparation, splitting, fitting, diagnostics, multicollinearity, prediction, and competing-model comparison without formal derivations.
- Examples
- Uses NBA regular-season wins, screen-assist points, deflections, and loose balls in full and reduced R models.
- What you can do after
- Apply it via R to fit, diagnose, compare, and interpret multiple linear models for continuous targets.
Grokking Statistics
A regression method that models a dependent variable as a linear function of one or more predictors.
- How it is used
- Predicts numeric dependent variables from one or more independent variables through a fitted linear function.
- How deep it goes
- Explains simple and multiple fits, coefficients, residuals, least squares, R², RMSE, and cautions while deferring optimization algorithms to libraries.
- Examples
- Uses temperature and sports drink sales, house-size predictors, house-price prediction, and a fitted scatterplot.
- What you can do after
- Apply it via SciPy linregress to fit, interpret, plot, and evaluate predictions while checking model assumptions.
Grokking AI Algorithms, Second Edition
Linear regression is a statistical modeling method that represents a continuous response as a linear function of one or more input variables.
- How it is used
- Introduces supervised prediction of continuous values using relationships among features.
- How deep it goes
- Works through fitting a one-input line using means, slopes and intercepts, then tests predictions on held-out data.
- Examples
- Uses diamond carat weights and prices to fit and test a prediction line, and contrasts regression with vehicle classification.
- What you can do after
- Build a one-input regression line from scratch and evaluate its predictions against held-out examples.
Read how Grokking AI Algorithms, Second Edition explains linear regression →
Machine Learning System Design
A linear model predicts an output as a linear combination of input features, optionally followed by a task-specific link or threshold function.
- How it is used
- Uses linear regression as a baseline for deciding whether a forecasting task needs a more complex model.
- How deep it goes
- Outlines baseline selection and residual diagnostics, including assumptions and potential failures, without deriving the regression algorithm.
- Examples
- Uses retail demand forecasting with lagged sales and rolling aggregates, alongside diagnostic plots comparing met and unmet assumptions.
- What you can do after
- Diagnose a fitted linear regression by checking residuals and identifying patterns that may warrant different features or models.
Read how Machine Learning System Design explains linear regression →
Deep Learning with PyTorch, Second Edition
A mathematical mapping that combines inputs linearly, used here as a component of neural network layers before activation.
- How it is used
- Introduces parameter fitting through temperature conversion, then uses linear operations as building blocks for neural networks.
- How deep it goes
- Works through a one-input, one-output model and its loss-based training, then connects linear transformations to nonlinear activations.
- Examples
- Uses paired thermometer and Celsius readings to fit a line, then compares the fit with a neural network.
- What you can do after
- Apply it via PyTorch to fit a temperature-conversion model and evaluate its predictions.
Read how Deep Learning with PyTorch, Second Edition explains linear regression →
Causal AI
A linear model represents an outcome as a linear or affine function of predictor variables and model parameters, typically with an error term.
- How it is used
- Estimates identified causal effects with a fitted treatment coefficient and represents proportional mechanisms in structural causal models.
- How deep it goes
- Defines parameter fitting, works through backdoor regression and its coefficient interpretation, and flags the limits of linear causal assumptions.
- Examples
- Uses a femur-length-to-height equation and a game-engagement regression with three confounders, an effect estimate, and a confidence interval.
- What you can do after
- Apply it via DoWhy to estimate an identified backdoor effect and interpret the treatment coefficient under a linearity assumption.
Machine Learning Algorithms in Depth
Linear regression is a regression model that represents a response as a linear combination of input features.
- How it is used
- Predicts continuous responses from feature vectors within supervised learning.
- How deep it goes
- Derives fitting from Gaussian noise and a coefficient prior, then implements gradient descent; stops short of posterior inference.
- Examples
- Uses a ridge-regression implementation and a California housing fit plotted against average room count.
- What you can do after
- Build a regularized linear regressor from scratch using gradient descent and apply it to housing-price data.
Read how Machine Learning Algorithms in Depth explains linear regression →
Data Without Labels
A linear transformation is a mapping that preserves vector addition and scalar multiplication.
- How it is used
- Predicts continuous targets from labeled historical data within supervised learning.
- How deep it goes
- Defines simple and multiple linear equations and illustrates a fitted line, without developing a training procedure.
- Examples
- Uses height to predict weight and names revenue and rainfall as other continuous targets.
- What you can do after
- Evaluate/understand it by identifying a continuous prediction task and reading a linear fit as a relationship between inputs and target.
Inside Deep Learning: Math, Algorithms, Models
A model whose output is a linear function of its inputs, used here to describe the early effective behavior of ReZero.
- How it is used
- Introduces a practical PyTorch model and training setup that can be extended to larger neural networks.
- How deep it goes
- Defines weights and bias, maps them to nn.Linear, and trains a fit; it stops short of nuanced loss-function comparisons.
- Examples
- Uses car-mileage prediction and a noisy one-dimensional dataset with a linear trend and oscillation.
- What you can do after
- Apply it via PyTorch's nn.Linear and MSELoss to fit and inspect a model for numeric prediction.
Read how Inside Deep Learning: Math, Algorithms, Models explains linear regression →
LLM Evaluation and Alignment, The Foundational Ideas
A regression model predicts a continuous numerical value from input features.
- How it is used
- Uses human-judgment data to convert translation features into an exact quality score.
- How deep it goes
- Defines prediction and loss, then connects them to a feed-forward regressor while deferring deeper implementation details.
- Examples
- Uses DA ratings, MQM scores, HTER values, and predicted outputs such as 78 or 78.5.
- What you can do after
- Apply it via COMET when absolute human quality scores are available for training.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains linear regression →
Grokking Bayes
Linear regression is a statistical modeling technique that represents the relationship between a response and one or more predictors with a linear function.
- How it is used
- Uses a linear relationship to predict responses and update uncertainty about feature effects.
- How deep it goes
- Develops a one-feature Bayesian model, closed-form updating, noise, overfitting, and briefly sketches multivariate extensions.
- Examples
- Uses house prices, square footage, synthetic noisy data, slope distributions, and prior-versus-posterior plots.
- What you can do after
- Apply it via closed-form calculations and Python snippets to estimate slopes, predict prices, and quantify slope uncertainty.
Bayesian Optimization in Action
Linear regression is a model that represents a response as a linear combination of input features and fitted coefficients.
- How it is used
- Encodes an expected objective trend inside Gaussian-process modeling and supports regression on continuous targets.
- How deep it goes
- Defines μ = wᵀx + b, explains coefficient roles and special cases, and demonstrates implementation without deriving coefficient fitting.
- Examples
- Uses housing prices, MNIST digit labels, advertising clicks, molecular energy, and drug effectiveness as regression targets.
- What you can do after
- Build it from scratch with GPyTorch by configuring LinearMean and pairing it with a covariance module.
Read how Bayesian Optimization in Action explains linear regression →
Advanced Algorithms and Data Structures
Linear regression is a supervised-learning technique that models the relationship between input variables and a real-valued dependent variable using a parameterized function.
- How it is used
- Uses labeled examples to estimate real-valued outputs such as goods’ prices.
- How deep it goes
- Explains model families, parameter fitting, complexity, and squared-error cost, but defers automated model selection.
- Examples
- Uses car characteristics and prices, plus constant, line, and quadratic models plotted against data.
- What you can do after
- Apply it to estimate a real-valued price from a new item’s measured characteristics.
Read how Advanced Algorithms and Data Structures explains linear regression →
Delivering Machine Learning Projects: From design to deployment
A linear model predicts an output as a linear combination of input variables and model parameters.
- How it is used
- Establishes baseline performance during early exploratory modelling and supports evaluation of later predictive models.
- How deep it goes
- Shows a quick implementation, correlated predictions, and explicit data assumptions, but stops short of solving the modelling problem.
- Examples
- Uses inventory demand, customer churn rates, limited datasets, and observed fluctuations as evaluation examples.
- What you can do after
- Evaluate model feasibility by producing baseline predictions and comparing them with observed fluctuations.
Timeless Algorithms, The Foundational Ideas
Linear regression is a statistical model that represents a continuous response as a linear function of predictors and model parameters.
- How it is used
- Models continuous responses to explain predictor relationships and predict responses for future observations.
- How deep it goes
- Defines the model, interprets coefficients and residuals, and explains squared-residual fitting and its likelihood connection.
- What you can do after
- Understand how to interpret regression coefficients, assess residual disagreement, and relate squared-error fitting to likelihood estimation.
Read how Timeless Algorithms, The Foundational Ideas explains linear regression →
Also covered in
- Build a Robo-Advisor with Python (From Scratch)
- Investing for Programmers
- Quantum Programming in Depth
- LLMs in Production
- Grokking Data Structures
- Evolutionary Deep Learning: Genetic algorithms and neural networks
- How Large Language Models Work
- Building Quantum Software with Python
- Deep Learning with JAX
- Grokking Algorithms, Second Edition
- Data Storytelling with Altair and AI
- Privacy-Preserving Machine Learning
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- objective functionAn objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
- machine learningMachine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
- classifierA classifier is a predictive model that assigns an input to one or more discrete classes or categories.
Related concepts
- confounderaddresses · A confounder is a variable associated with both an explanatory or treatment variable and an outcome, whose influence can distort their apparent relationship or estimated causal effect.
- inputproduces · An input is information, a value, or an action supplied to a system, program, or computation for processing or to influence its behavior.
- supervised learningkind of · Supervised learning is a machine-learning approach that learns predictive relationships from examples paired with known target outputs or labels.
- classifiercontrasts with · A classifier is a predictive model that assigns an input to one or more discrete classes or categories.
Questions about linear regression
What is linear regression?
Linear regression is a statistical method that models a numeric response as a linear function of one or more predictor variables with fitted parameters.
Which Manning books explain linear regression?
Causal Inference for Data Science, Grokking Machine Learning, Effective Data Analysis, Statistics Every Programmer Needs, Math and Architectures of Deep Learning, Build AI Drug Discovery Pipelines, Grokking Machine Learning, Second Edition, Experimentation for Engineers and 20 more.
What should I know before learning linear regression?
Manning books that teach linear regression usually introduce language model, objective function, machine learning and classifier first.
What is the difference between linear regression and classifier?
Linear regression is a statistical method that models a numeric response as a linear function of one or more predictor variables with fitted parameters. A classifier is a predictive model that assigns an input to one or more discrete classes or categories.
What kinds of linear regression are there?
Kinds of linear regression include multiple linear regression and simple linear regression.
What is linear regression a kind of?
Linear regression is a kind of supervised learning. Supervised learning is a machine-learning approach that learns predictive relationships from examples paired with known target outputs or labels.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



























