Logistic regression estimates the probability of a class from weighted input features. Its linear combination becomes a soft binary decision through the sigmoid, or logistic, function: an S-shaped curve that acts like a tunable threshold. section 5.1.4 section 5.2
A single neuron using the logistic function is mathematically equivalent to scikit-learn’s LogisticRegression model; the main difference is how the two models are trained. Changing the coefficient changes the curve’s slope, while changing the intercept or phase shifts its position. section 5.1.4 section 5.2
In natural-language applications, logistic regression commonly works with:
- Bag-of-words counts or TF-IDF vectors produced after tokenization. section 5.1.4
- Character n-grams, which let a model represent unfamiliar names and other out-of-distribution strings. section 5.2.4
- Binary logic gates or conditional decisions, such as filtering potentially sensational links. section 5.2.1
Clickbait classification
A headline-length feature can feed a logistic regression classifier that estimates whether a headline is fake or authentic. The model’s slope describes how sharply that probability changes around its decision boundary, rather than proving that length alone determines truthfulness. section 5.2.1
Names and character features
Aggregated name counts across years and regions create training examples for predicting the most common recorded sex category. TfidfVectorizer can represent each name with character 1- to 3-grams, supporting predictions for names absent from the training data. section 5.2.4
>>> def logistic(x, w=1., phase=0, gain=1): ... return gain / (1. + np.exp(-w * (x - phase)))
Run the compact Python logistic function and vary its weight, phase, and gain to inspect how the probability curve changes. section 5.1.4
>>> import pandas as pd
>>> import numpy as np
>>> import seaborn as sns
>>> sns.set_style()
>>> xy = pd.DataFrame(np.arange(-50, 50) / 10., columns=['x'])
>>> for w, phase in zip([1, 3, 1, 1, .5], [0, 0, 2, -1, 0]):
... kwargs = dict(w=w, phase=phase)
... xy[f'{kwargs}'] = logistic(xy['x'], **kwargs)
>>> xy.plot(grid="on", ylabel="y")
Plot several logistic curves to compare coefficient and phase effects visually. section 5.1.4
