Why might an assistant refuse a dangerous request or steer a conversation elsewhere? A guardrail is a learned constraint on whether and how a model responds. It emerges during alignment rather than from a hand-written list of forbidden topics. section 2.2.3 section 2.3
Guardrails develop through a simple training pattern:
- Human reviewers rank candidate responses by qualities such as helpfulness, clarity, safety, and tone. section 2.2.3
- A reward model learns those preferences, and updates make highly rated responses more likely. section 2.2.3
- Behavioral guardrails shape communication, while safety guardrails shape what the model will and will not do. section 2.2.3
A refusal is learned behavior
A user asks for instructions that could cause harm. Instead of supplying them, the assistant gives a polite refusal or redirects the conversation. The model is not necessarily checking a blocklist; it is exhibiting a response pattern reinforced during alignment. section 2.2.3
Figure 2.8 The RLHF loop applied to a single prompt. The model generates two candidate responses; human reviewers rank them by helpfulness and clarity; a reward model learns that preference; the LLM is updated to make better responses more likely. The process repeats until the model's behavior aligns with human expectations.

The RLHF loop shows how reviewers rank alternative responses, a reward model learns the preference, and the LLM is updated repeatedly. section 2.2.3
Guardrails are more flexible than a fixed blocklist, but they are imperfect: they reflect feedback providers’ priorities and blind spots, vary between models, and may fail on inputs the training process did not anticipate. section 2.3






















