liveBook · Manning

Language model

As explained in LLM Evaluation and Alignment, The Foundational Ideas

A large language model is a neural language model trained on large text corpora to generate or evaluate natural-language content.

A language model is a probabilistic text generator that predicts likely next tokens from patterns learned in data. Its outputs are free-form: one prompt can produce many valid answers, while statistically plausible text can still be factually wrong. section 1.0 section 5.4.5

Because language models learn patterns rather than following only explicit programs, they can develop capabilities and behaviors that were not directly specified. Evaluation measures what the model does; alignment steers behavior toward human goals, policies, principles, or a consistent persona. section 1.0 section 1.2

Analyze language-model behavior through complementary lenses:

  • Evaluate accuracy, reliability, helpfulness, coherence, tone, and edge-case behavior rather than relying only on unit tests or loss curves. section 1.0
  • Align responses with explicit policies, higher-level principles, human goals, or a chosen persona. section 1.2
  • Monitor hallucinations because mitigation can reduce their rate but cannot currently eliminate them. section 5.4.5

Preference learning for responses

In RLHF, a reward model compares complete responses rather than short action segments. Human preference labels train the reward model, which then scores language-model outputs; the longer response introduces a difficult credit-assignment problem. section 7.4.3

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines probabilistic next-token generation and discusses evaluation, hallucination, alignment, and reward modeling, but does not detail model architecture.
Examples
Uses summarization, full-response preference comparisons, pirate roleplay, sycophancy, and confident hallucinations as concrete cases.
What you can do after
Evaluate model behavior against accuracy, reliability, edge cases, and human goals, then identify alignment or mitigation needs.
How it connects
Connects language models to evaluation, alignment, hallucination detection, reward models, RLHF, and retrieval-augmented generation.

Also explained in

Compare how 89 books explain language model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.