Intercept

As explained in Data Without Labels

A bias term is a learnable additive neural-network parameter that shifts a unit's pre-activation independently of its weighted inputs.

In a neural network, the intercept is the learnable bias term: an additive parameter that shifts a unit’s pre-activation independently of its weighted inputs. It is analogous to the constant c in y = mx + c, while weights play the role of slopes. section 8.2.2

Use the intercept to understand how a perceptron and a trained network make predictions:

• A perceptron combines input values with weights and a bias before applying its activation function. section 8.2.2
• Weights and biases are trained parameters; learning searches for values that improve prediction accuracy across network layers. section 8.3
• During feed-forward propagation, hidden layers process data using activation functions, weights, and bias terms before producing a prediction. section 8.3.2

Face identification

A face-recognition network learns weights and bias values from labeled face images. During feed-forward propagation, those parameters help transform image attributes into a probability that the face belongs to a particular person. section 8.2.1 section 8.3.2

Figure 8.3 A neuron gets the inputs, processes them using mathematical functions, and then generates the output.

A neuron diagram helps connect the intercept to the flow from inputs through mathematical processing to an output. section 8.2.2

How Data Without Labels covers it

How deep it goes

Defines the bias through a linear-function analogy and feed-forward role, but defers detailed activation-function treatment.

Examples

Uses face identification, labeled face images, threshold behavior, and neuron diagrams to illustrate the parameter.

What it assumes

Assumes familiarity with inputs and linear equations while introducing perceptrons, weights, biases, layers, and activation functions.

What you can do after

Evaluate/understand how bias values participate in perceptron computation, parameter training, and feed-forward prediction.

How it connects

Connects the intercept to perceptrons, weights, activation functions, layers, and feed-forward propagation.

liveBookmanning.com

Read Data Without Labels on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningThe value of a linear function when its input features are zero, corresponding to where the line crosses the y-axis.
• 

  liveBookmanning.com
  Statistics Every Programmer NeedsAn intercept is the predicted value of a response variable when all predictors in a linear equation equal zero.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningA learned additive parameter that shifts model output independently of the inputs.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA bias is a constant input signal, conventionally equal to one, that allows a neuron to shift its activation independently of the other inputs.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionA neural-network bias is a learned additive parameter that shifts a layer's computed output independently of its weighted inputs.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthA bias parameter is a learned additive parameter that shifts a neural-network layer's preactivation values.

Compare how 7 books explain intercept →

Related concepts

• parameterkind of · A model parameter is a numerical value, such as a weight or bias, that is learned or set to determine how a model computes outputs.
• thresholdequivalent to · A threshold is a boundary value against which a quantity is compared to determine whether a condition is met or an action or decision is triggered.
• classification scoreequivalent to · A classification score is a real-valued output representing a classifier’s support, confidence, or signed strength for a predicted class.
• line equationpart of · A line equation is an equation linear in its variables that represents a straight line in two dimensions or a linear relationship in higher dimensions.
• biasequivalent to · Bias is a systematic deviation or tendency in data, measurements, estimates, judgments, or outputs that skews them relative to a relevant target or impartial baseline.
• alphaequivalent to · Alpha is investment performance beyond a relevant benchmark or the return explained by specified systematic risk factors.

This explainer was generated by Manning's knowledge graph from the text of Data Without Labels. The section links point to the parts of the book it draws on.