liveBook · Manning

Intercept

As explained in Natural Language Processing in Action, Second Edition

A bias is a constant input signal, conventionally equal to one, that allows a neuron to shift its activation independently of the other inputs.

An intercept, called a bias weight in a neural network, is an always-on input signal. Setting x0 to 1.0 lets w0 shift the neuron’s activation independently of the other inputs, just as b shifts linear regression output. section 5.1.5

Adding the bias to a perceptron

For token counts green=1, egg=2, ham=3, and spam=4, prepend 1 to the input vector and place w0 first in the weight vector. The weighted sum then includes w0 * 1.0 alongside the feature-weight products; a threshold converts the result into a prediction.

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Explains the intercept-to-bias correspondence and weighted-sum implementation, stopping at feed-forward calculation rather than learning procedures.
Examples
Demonstrates bias handling with a four-token count vector, randomly initialized weights, a weighted sum, and a zero threshold.
What it assumes
Assumes basic familiarity with linear-regression slope and intercept while relating those terms to neural-network weights.
What you can do after
Build a perceptron feed-forward calculation by prepending a constant one and pairing it with a dedicated trainable weight.
How it connects
Connects intercepts with bias weights, linear regression, logistic regression, perceptrons, neurons, and thresholding.

Also explained in

Compare how 7 books explain intercept →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.