liveBook · Manning

Logistic regression

As explained in Machine Learning Algorithms in Depth

A probabilistic classification model that maps inputs to class probabilities using a logistic or softmax transformation of a linear score.

Logistic regression is a probabilistic classification model that computes the probability of a class label from input data. For multiple classes, it uses softmax to transform class scores into probabilities that sum to one; for binary classification, it uses a Bernoulli model and the logistic function. section 5.4 section 5.8

The training procedure follows these ideas:

  • Compute posterior class probabilities using Bayes rule and class priors. section 5.4
  • Represent multiclass scores with softmax, whose outputs form a probability distribution. section 5.4
  • Maximize log likelihood, equivalently minimize negative log likelihood. section 5.4
  • Update parameters with stochastic gradient descent using a learning-rate schedule satisfying Robbins–Monro conditions. section 5.4

Synthetic binary classification

A from-scratch Python implementation generates two-dimensional samples from two Gaussian clusters, assigns labels 1 and -1, trains with SGD, and visualizes both training cost and the resulting decision boundary.

How Machine Learning Algorithms in Depth covers it

How deep it goes
Derives posterior probabilities, likelihood, gradients, and SGD updates before showing an implementation and visual results.
Examples
Demonstrates two Gaussian feature clusters, SGD training, cost reduction, and a learned decision boundary.
What it assumes
Uses Bayes rule, probability, Bernoulli distributions, differentiation, and gradient-based optimization.
What you can do after
Build it from scratch with stochastic gradient descent and visualize its cost and decision boundary.
How it connects
Connects Bayes rule, Bernoulli distributions, softmax, negative log likelihood, and stochastic gradient descent.
Cover of Machine Learning Algorithms in Depth
liveBookmanning.com
Read Machine Learning Algorithms in Depth on liveBook Get the book at manning.com

Also explained in

Compare how 12 books explain logistic regression →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Algorithms in Depth. The section links point to the parts of the book it draws on.