A persona vector is a reusable linear direction in a model’s residual stream that represents a behavioral trait. It provides an inference-time alternative to shaping personality through demonstrations or retraining, allowing practitioners to control and monitor character at the representation level. section 17.1.1
A practical persona-vector pipeline consists of these operations:
- Define a trait in natural language, such as “sycophancy: excessive agreeableness and flattery.” section 17.1.1
- Generate contrastive system-prompt pairs that elicit or suppress the trait, then collect the target model’s responses. section 17.1.1
- Average residual-stream activations over response tokens at a selected layer and subtract the trait-negative mean from the trait-positive mean: vℓ = mean(S⁺) − mean(S⁻). section 17.1.1
- Intervene during inference by applying the direction at selected layers, using a signed scale such as hℓ ← hℓ + α·vℓ; the illustrated pipeline uses subtraction for steering. section 17.1.1
Steering a behavioral trait
Suppose contrastive prompts produce responses that respectively express and suppress sycophancy. Differencing their averaged activations yields a sycophancy direction that can later be reused to move model behavior toward or away from that trait without retraining. section 17.1.1
Figure 17.1 The persona vector extraction and intervention pipeline. Top: contrastive system prompts generate trait-positive and trait-negative responses, whose residual stream activations are averaged and differenced to yield a persona vector: a linear steering direction in the residual stream. Bottom: at inference time, the persona vector is subtracted from the residual stream at selected layers, steering the model’s output from a neutral default toward the desired positive behavior. Adapted from Chen et al. (2025) [3].
The pipeline diagram connects contrastive prompt generation, residual-stream averaging, vector extraction, and inference-time steering in one view. section 17.1.1
