liveBook · Manning

Contrastive prompt pair

As explained in Rearchitecting LLMs

A paired-prompt method compares prompts that differ in one controlled attribute to isolate its effect on model behavior.

A contrastive prompt pair contains two otherwise identical prompts that differ in one controlled attribute. Comparing their outputs or activations helps isolate how that attribute affects model behavior and identify neurons with different responses. section 11.0 section 11.1

Changing one demographic attribute

For a race comparison, describe the same pedestrian in the same situation, changing only the attributed race from “white” to “Black.” A gender comparison changes “boy” to “girl” in a fighter-pilot prompt. section 11.1 section 11.3

Listing 11.15 A gender-contrastive prompt pair
PILOT_PROMPT_MAJORITY = (
    "The boy said he wanted to be a fighter pilot "
    "when he grew up. The adults felt"
)
 
PILOT_PROMPT_STEREOTYPED = (
    "The girl said she wanted to be a fighter pilot "
    "when she grew up. The adults felt"
)
 
tokens_pilot_majority = tokenizer(PILOT_PROMPT_MAJORITY, return_tensors="pt")["input_ids"]
tokens_pilot_stereotyped = tokenizer(PILOT_PROMPT_STEREOTYPED, return_tensors="pt")["input_ids"]
print(f"Tokens (pilot majority)   : {tokens_pilot_majority.shape[1]}")
print(f"Tokens (pilot stereotyped): {tokens_pilot_stereotyped.shape[1]}")

The paired prompts and token-count check make the controlled comparison concrete. section 11.3

How Rearchitecting LLMs covers it

How deep it goes
Defines the controlled-pair method and links activation differences to neuron ranking, but leaves judgments about what counts as bias to broader processes.
Examples
Compares white and Black pedestrian prompts and boy and girl fighter-pilot prompts.
What it assumes
Assumes neuron-analysis tools and prompts known to produce different responses, while defining the controlled-pair structure explicitly.
What you can do after
Diagnose model differences by comparing matched-prompt activations and identifying neurons with the largest differences.
How it connects
Connects controlled prompts to neuron activation analysis, targeted neuron scaling, and BBQ bias benchmarking.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.