liveBook · Manning

Alignment

As explained in AI Model Evaluation

Alignment is the degree to which a model's behavior, reasoning, and outputs match specified evaluation objectives and human judgment.

Alignment is the broader goal of making an LLM judge’s behavior, reasoning, and outputs match evaluation objectives and human judgment. Calibration supports this goal by anchoring scores to labeled gold-standard examples. section 9.6.1

Align an evaluation setup through several complementary practices:

  • Design clear prompts and specify the evaluation rubric. section 9.6.1
  • Select models trained with objectives appropriate to the evaluation task. section 9.6.1
  • Validate judgments continuously against human ratings or historical baselines. section 9.6.1

Movie recommender judge

A movie recommendation evaluator receives a structured prompt and a few human-scored examples showing what scores such as 5 and 2 represent. These gold examples reduce arbitrary scoring, limit subjective drift, and make the evaluator’s reasoning more traceable. section 9.6.1

How AI Model Evaluation covers it

How deep it goes
Defines alignment broadly and identifies prompt design, rubrics, model choice, calibration, and human validation as contributing practices.
Examples
Uses a movie recommender judge, structured prompts, score anchors, gold-standard examples, human ratings, and historical baselines.
What you can do after
Evaluate and improve an LLM judge by calibrating scores and validating them against human ratings.
How it connects
Connects alignment to calibration, prompt design, rubric specification, model selection, and human evaluation.

Also explained in

Compare how 6 books explain alignment →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.