liveBook · Manning

Language model

As explained in AI Governance

A large language model is a machine-learning model trained on large text corpora to process and generate natural language.

A language model interprets natural language for meaning. In a prompt that asks for an email summary, the instruction and email arrive as plain text; the model infers which is command and which is data from context, not a reliably enforced technical boundary. Filters can block obvious attacks, but they remain workarounds rather than hard boundaries. section 4.0

A model’s behavior is shaped by its weights: numerical parameters adjusted during training. Because those parameters encode behavior, inspecting a model file cannot reveal every vulnerability embedded in what the model has learned. section 4.4.1

Account for these risks when deploying or adapting language models:

  • Treat prompt filters as bypassable safeguards, not as a guaranteed separation between instructions and data. section 4.0
  • Check safety behavior after fine-tuning: adapting a model can weaken refusals, even when the added dataset is benign. section 4.4.1

Fine-tuning can change safety behavior

A reported study found that fine-tuning GPT-3.5 Turbo on 10 adversarial examples weakened its safety guardrails; ordinary datasets also degraded safety, though less dramatically. section 4.4.1

How AI Governance covers it

How deep it goes
Explains weight-based behavior and fine-tuning-related safety degradation, but offers no method for inspecting model weights.
Examples
Uses an email-summary prompt, the EchoLeak attack, and fine-tuning studies to illustrate model risks.
What you can do after
Understand why prompts resist rigid command-data separation and why fine-tuning can weaken safety behavior.
How it connects
Connects language models to prompt injection, guardrails, fine-tuning, and neural-network weights.

Also explained in

Compare how 89 books explain language model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.