Weight

As explained in Introduction to Generative AI, Second Edition

A model weight is a learned numerical parameter that determines the strength of a computational connection in a machine-learning model.

In a neural network, a weight represents the strength of a connection between nodes. Each node combines weighted inputs and adds a bias; training iteratively adjusts weights and biases to perform well against the model’s objective. A model’s parameter count refers to its learned weights and indicates the complexity it can handle. section 1.1

For a large language model, learned weights are an end product of training. Access to them can enable reproduction of the model, making the protection of proprietary, unreleased weights a security concern. section 10.29

Llama 1 weight leak

After Meta’s Llama 1 weights leaked, users quickly produced a high-quality language model based on Llama 1. section 10.29

Figure 10.5 The three pillars of voluntary commitments made to the White House by leading AI companies

The figure summarizes voluntary AI commitments, including security commitments to protect proprietary, unreleased model weights. section 10.29

How Introduction to Generative AI, Second Edition covers it

How deep it goes

Explains weights as connection strengths, their iterative adjustment, and their role in parameter counts, but defers the underlying mathematics.

Examples

Uses the leak of Meta’s Llama 1 weights and rapid reproduction of a model based on it as a security example.

What you can do after

Understand how training sets connection strengths and why access to model weights can enable model reproduction.

How it connects

Links weights to neural networks, training, bias terms, model parameters, and large language models.

liveBookmanning.com

Read Introduction to Generative AI, Second Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Data Without LabelsA weight is a learned numerical parameter that scales the contribution of an input or intermediate activation in a neural network.
• 

  liveBookmanning.com
  Fabulous Adventures in Data Structures and AlgorithmsAn integer weight is a whole-number quantity representing the relative contribution of an outcome to a weighted distribution.
• 

  liveBookmanning.com
  Kubernetes in Action, Second EditionA weight is a routing value that determines a backend's proportional share of requests relative to the weights of other backends.

Compare how 4 books explain weight →

Related concepts

• fine-tuningused by · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
• deep learningpart of · Deep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
• neural networkpart of · A neural network is a computational model of interconnected, parameterized units that learns transformations from inputs to outputs.
• parameterequivalent to · A model parameter is a numerical value, such as a weight or bias, that is learned or set to determine how a model computes outputs.
• linear regressionused by · Linear regression is a statistical modeling method that estimates a continuous response as a linear combination of predictor variables and fitted coefficients.

This explainer was generated by Manning's knowledge graph from the text of Introduction to Generative AI, Second Edition. The section links point to the parts of the book it draws on.