Can a classifier say more than “happy” or “sad”? Logistic regression produces a score between 0 and 1 instead of only a class label. A strongly happy sentence might score near 1, a strongly sad one near 0, and a neutral sentence near 0.5. It is also called a logistic classifier or continuous perceptron. section 6.0 section 6.1
Picture a line separating two classes. A discrete perceptron assigns 0 or 1 according to which side a point falls on. A logistic classifier scores every point: points on the line score 0.5, while points farther into either class’s region score closer to 0 or 1. Using 0.5 as a cutoff turns those scores back into class predictions; that cutoff is common but adjustable. section 6.1 section 6.0
Figure 6.3 Left: The step function used to build discrete perceptrons. It outputs a value of 0 for any negative input and a value of 1 for any input that is positive or zero. It has a discontinuity at zero. Right: The sigmoid function used to build continuous perceptrons. It outputs values less than 0.5 for negative inputs and values greater than 0.5 for positive inputs. At zero, it outputs 0.5. It is continuous and differentiable everywhere.
Compare the perceptron’s abrupt step with the smooth sigmoid that gives logistic regression its continuous output. section 6.1
To calculate a score, multiply each feature by its weight, add the results and a bias, then apply the sigmoid function: ŷ = σ(w₁x₁ + ⋯ + wₙxₙ + b). The sigmoid maps a negative input below 0.5, zero to 0.5, and a positive input above 0.5. Unlike the step function, it changes smoothly. The score can be interpreted as the model’s probability for label 1. section 6.1 section 6.3
Training aims to give label-0 points low scores and label-1 points high scores. Log loss measures how far a prediction falls short: for label y and prediction ŷ, it is −y ln(ŷ) − (1 − y) ln(1 − ŷ). A confident prediction for the wrong label incurs a large loss. Starting with weights and a bias, the logistic regression algorithm repeatedly picks a labeled point, adjusts the model, and tracks loss. Its update uses the difference y − ŷ, so even correctly classified points can help improve their scores; this contrasts with the perceptron update, which leaves them alone. section 6.1 section 6.2 section 6.3
From word counts to review sentiment
Suppose each sentence is represented by counts of two words and labeled happy or sad. Plotting those counts makes the boundary and its changing scores visible; coding the sigmoid, log loss, and repeated updates fits a classifier to the labeled points. For a larger task, Turi Create turns IMDB reviews into word-count features, trains a logistic classifier on positive and negative labels, and requests probability predictions. Word weights can then be inspected through the model’s coefficients. section 6.3 section 6.4
To check whether logistic regression suits a dataset, evaluate it on data set aside from training rather than relying only on fitted scores. One Titanic survival workflow trains Scikit-Learn’s LogisticRegression alongside other classifiers and compares validation accuracy and F1 score. For a smaller hands-on check, calculate a point’s prediction and log loss, make one logistic update, and see whether its loss falls. section 13.4 section 6.7
