liveBook · Manning

Refusal behavior

As explained in LLM Customization and Fine-Tuning

A model's act of declining to fulfill a request, often because complying would be inappropriate or unsafe.

Refusal behavior is a model’s decision to decline a request when compliance would be inappropriate or unsafe. A model can retain strong task performance after adaptation yet lose refusals it previously made, making independent safety testing a deployment requirement. section 7.9 section 5.4.1

In black-box distillation, a student trained only on benign teacher outputs has no examples connecting harmful requests to refusals. A fluent, on-topic response can still be unsafe, so ordinary quality checks for length and coherence are not enough. Include complete harmful-prompt and teacher-refusal pairs, verify that the teacher refuses appropriately, and retest the student. section 7.8.1 section 7.8.3

Refusing while offering a safe next step

An IT support assistant asked for an administrator password declines to share it and directs the user to the approved access-request process. The training example pairs the request with the desired refusal and alternative. section 5.1.7

{"messages": [
  {"role": "system",    "content": "You are Contoso's internal IT support assistant."},
  {"role": "user",      "content": "What is the admin password for the finance file share?"},
  {"role": "assistant", "content": "I can't share admin passwords. To get access, open the IT portal, go to Access > File Shares, and submit a request for your manager to approve."}
]}

This training record shows the full user request and assistant response, including a refusal and an approved access path. section 5.1.7

For LoRA data, deliberately overrepresent rare, high-cost safety refusals rather than matching their production frequency. One recommendation is 50–100 explicit refusal examples, or 10%–20% of the dataset, alongside informative answers, clarifications, and other response types. For a distillation regression, a smaller first intervention is to add 10–20 verified teacher refusal examples. section 5.1.7 section 7.8.3

Test and respond to regressions before deployment:

  • Compare the base or teacher model with the adapted model on sensitive prompts in a controlled environment. An unsafe answer where the earlier model refused is a serious regression. section 5.4.1
  • Set minimum pass rates before training: 100% for harmful-request refusal, 80% or more for uncertainty and general knowledge, and 90% or more for bias detection. Treat the check as a deployment gate. section 7.8.3
  • If adding refusal pairs is insufficient, apply safety-specific fine-tuning with a LoRA adapter. An input/output classifier can also screen unsafe outputs, but it does not repair the student’s behavior. section 7.8.3

Preference optimization can also reduce refusal probabilities, so rerun safety tests after DPO. Add preference pairs with refusal as the chosen response and harmful compliance as the rejected response, then retest. If supervised fine-tuning data lacked refusals, adding them there is more reliable than relying on a modest number of DPO safety pairs alone. section 8.9

No regression does not mean safe

In one four-prompt check, both the supervised model and the DPO model passed only one test. DPO did not worsen the score, but neither model refused the phishing request; existing failures still needed correction. section 8.9

How LLM Customization and Fine-Tuning covers it

How deep it goes
Explains failure mechanisms and gives data, testing, and mitigation steps, including deployment thresholds.
Examples
Uses an IT password request and a four-prompt safety check that includes a phishing request.
What you can do after
Diagnoses safety regressions by comparing models and applying refusal-data, fine-tuning, or filtering mitigations.
How it connects
Connects refusal behavior to knowledge distillation, supervised fine-tuning, preference optimization, safety alignment, and regression testing.

Also explained in

Compare how 2 books explain refusal behavior →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.