Logistic regression

As explained in Ensemble Methods for Machine Learning

Logistic regression is a statistical classification model that estimates class probabilities by applying a logistic function to a linear combination of input features.

Logistic regression classifies two outcomes, such as 0 or 1, by passing a linear combination of features through the sigmoid function. The result is a probability between 0 and 1. Its linear predictor represents log-odds: ln(p/(1−p)) = β0 + β′x. The Bernoulli distribution models the yes/no target, making the method suitable for binary classification. section 7.1.3

Read a fitted model through its additive feature weights:

• A positive weight increases the predicted log-odds and therefore increases the odds of the positive class. section 9.1.3
• Increasing a feature by one unit multiplies the odds by the exponential of its weight; a weight of 0.5 multiplies them by about 1.65. section 9.1.3

From diagnosis to implementation

For diabetes diagnosis, age and blood-glucose measurements can form an additive predictor of positive-diagnosis log-odds. A practical breast-cancer workflow standardizes the data, splits it into training and test sets, fits scikit-learn's LogisticRegression with L1 penalty and the saga solver, predicts test labels, and examines feature weights. section 9.1.3

Figure 7.5 Logistic regression (left) fits 0/1-valued targets by assuming that the spread of the targets can be modeled by the discrete-valued Bernoulli distribution (right). Observe how the prediction probabilities (the heights of the bars) of Class 0 and Class 1 change with the data.

Use the figure to see how class-0 and class-1 prediction probabilities change across a one-feature dataset under a Bernoulli model. section 7.1.3

How Ensemble Methods for Machine Learning covers it

How deep it goes

Explains the logit, sigmoid, Bernoulli assumption, additive weights, and odds interpretation, then demonstrates library training.

Examples

Uses diabetes diagnosis, breast-cancer diagnosis, a one-feature probability figure, and a scikit-learn training listing.

What it assumes

Assumes familiarity with features, labels, regression, probability, and exponential notation while reinforcing binary-label modeling.

What you can do after

Apply it via scikit-learn, estimate binary-class probabilities, and interpret feature effects through changes in odds.

How it connects

Connects generalized linear models, linear regression, Poisson regression, sigmoid functions, log-odds, odds, and Bernoulli distributions.

liveBookmanning.com

Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA perceptron-like model that outputs a continuous value between 0 and 1 rather than a discrete class label.
• 

  liveBookmanning.com
  Statistics Slam DunkLogistic regression is a statistical model that predicts the probability of a categorical outcome, typically by applying a logistic function to a linear predictor.
• 

  liveBookmanning.com
  Grokking StatisticsA regression method that models the probability of a categorical outcome using a logistic link function.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionLogistic regression is a statistical classification model that estimates the probability of a class from a weighted combination of input features.
• 

  liveBookmanning.com
  Causal Inference for Data ScienceLogistic regression is a statistical model that estimates the probability of a categorical, often binary, outcome using a logistic function of input variables.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AILogistic regression is a regression model that estimates class probabilities using a logistic link function.
• 

  liveBookmanning.com
  Effective Data AnalysisLogistic regression is a statistical model for estimating the relationship between a categorical outcome and one or more predictors.
• 

  liveBookmanning.com
  Data Without LabelsA classification method that models class-event probabilities using a logistic or sigmoid transformation of input variables.
• 

  liveBookmanning.com
  Starting Data Analytics with Generative AI and PythonLogistic regression is a regression method that models the probability of a categorical outcome, commonly a binary outcome.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthA probabilistic classification model that maps inputs to class probabilities using a logistic or softmax transformation of a linear score.
• 

  liveBookmanning.com
  The Art of AI Product DevelopmentLogistic regression is a statistical classification model that estimates class probabilities using a logistic function of input features.

Compare how 12 books explain logistic regression →

Related concepts

• maximum likelihood estimationuses · Maximum likelihood estimation is a statistical method that selects parameter values maximizing the likelihood of observed data under a specified probabilistic model.
• classifierkind of · A classifier is a predictive computational model that assigns inputs to one or more discrete classes or categories.
• sigmoiduses · A sigmoid is a smooth, bounded, S-shaped function that maps real-valued inputs between two limiting values, commonly zero and one.
• generalized linear modelkind of · A generalized linear model combines a linear predictor, a link function, and a response distribution from the exponential family.
• linear regressioncontrasts with · Linear regression is a statistical modeling method that estimates a continuous response as a linear combination of predictor variables and fitted coefficients.

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.