liveBook · Manning

Model weight

As explained in Machines that Think

A weight is an adjustable numerical parameter that scales the contribution of an input to a computational unit.

A model weight is an adjustable numerical parameter that scales an input’s contribution to a computational unit. In a neural network, weights shape how signals combine and how strongly they influence the output. section 3.1.1 section 3.1.3

An artificial neuron multiplies each input by its corresponding weight and adds the results. A bias shifts the combined signal, and an activation function transforms it into the neuron’s output. Many such units working together can represent patterns beyond the reach of a single neuron. section 3.1.1

Figure 3.1 Structure of an artificial neuron. Each input is scaled by a weight, summed with a bias, and passed through an activation function. This simple pipeline is the basic computational unit from which neural networks are built.

The diagram traces inputs through weights, a bias, and an activation function. section 3.1.1

During training, weights change in response to examples. Backpropagation identifies how each weight contributes to the loss, and an optimizer uses that information to adjust parameters. Over many iterations, the network develops patterns of influence that shape its responses. section 3.1.3 section 3.6

Estimating spam likelihood

A neuron could combine signals such as keyword frequency, number of links, message length, and whether the sender is known. Different weights make these signals contribute differently to an estimate of spam likelihood. In practice, inputs can be abstract numerical representations rather than features people can readily name. section 3.1.3

Learned knowledge is distributed across many weights rather than stored as a complete concept in any one of them. This distribution helps networks capture subtle patterns, but also makes their internal computations difficult to trace. Explainable AI and mechanistic interpretability investigate how model behavior arises. section 3.1.3

In a language model, learned weights transform the current input and influence which continuations are likely. They remain fixed during ordinary inference; the prompt supplies immediate context. If information is neither encoded in the parameters nor supplied in the input, it cannot influence the answer. section 6.4.1

How Machines that Think covers it

How deep it goes
Explains weighted neuron computations and distributed learning, then summarizes how backpropagation and optimizers adjust weights.
Examples
Illustrates weighted influence with spam-likelihood signals, including keyword frequency and whether the sender is known.
What you can do after
Evaluate how learned weights influence outputs and why their distributed organization can make model behavior difficult to trace.
How it connects
Connects weights to bias, activation functions, loss, backpropagation, and distributed representations.

Also explained in

Compare how 8 books explain model weight →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.