liveBook · Manning

Parameter

As explained in Timeless Algorithms, The Foundational Ideas

A parameter is a quantity that determines aspects of a model’s behavior.

A parameter is a quantity that determines an aspect of a model’s behavior. In a neural network, weights and biases are adjustable parameters: they govern how information moves through the layers and change as the network learns from data. section 13.2.1 section 11.2

In logistic regression, the intercept β₀ and coefficients β₁ through βₖ determine the linear predictor z. Each coefficient weights a predictor variable; the resulting z is the log-odds of the positive outcome. The logistic function then maps z to a probability. Changing the coefficients changes that probability for a given input. section 3.3.2

z=β0+β1x1+β2x2+⋯+βkxk

The linear predictor combines the intercept with the weighted predictor variables. section 3.3.2

Coefficients and a predicted probability

With β₀ = −6 and β₁ = 1.2, an input x = 5 gives z = −6 + 1.2 × 5 = 0. The corresponding positive-class probability is 0.5; at x = 2, it is approximately 0.027. section 3.3.2

Neural-network training repeatedly makes a prediction, measures its error, and adjusts parameters. Backpropagation calculates the loss gradient with respect to each weight and bias; an optimizer uses those gradients to update them. A single loss function guides adjustments throughout the network, including the layers that produce internal representations. section 11.2 section 11.2.6

Parameter count affects representational capacity, not what a model necessarily learns. A deeper or wider network can represent a broader range of relationships, but learning them depends on training data and compute. Highly flexible models can also fit patterns that do not generalize beyond their training sample. section 13.2.1

Not every parameter has the same role. In a soft-margin support vector machine, the specified penalty parameter C controls the cost assigned to margin violations. It influences which boundary the optimizer selects, rather than functioning as one of the fitted regression coefficients described above. section 8.3.4 section 3.3.2

A penalty changes the preferred boundary

For three illustrative candidate boundaries, C = 0.10 gives the wide boundary the lowest objective value; C = 1 favors the moderately adjusted boundary; and C = 100 favors the aggressive boundary. The values illustrate the trade-off and are not fitted estimates from the plotted observations. section 8.3.4

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains coefficients and neural-network updates mathematically, and compares illustrative soft-margin objectives under different penalties.
Examples
Illustrates logistic probabilities for specified coefficients and compares three candidate boundaries under different slack penalties.
What you can do after
Evaluate and understand how parameter values affect predictions, training, model capacity, and boundary selection.
How it connects
Connects parameters to logistic regression, maximum likelihood, backpropagation, cross-entropy, and representational capacity.

Also explained in

Compare how 41 books explain parameter →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.