A sparse autoencoder (SAE) reconstructs activations from a model layer through a much wider intermediate space. A training penalty encourages only a few expanded dimensions, or features, to activate per input, aiming to separate mixed concepts into more interpretable features. section 10.4 section 11.5.1
This addresses superposition, in which an original neuron responds to multiple topics. Once a concept feature is identified, modifying or clamping its activation during inference can steer the model toward that concept. section 10.4 section 11.5.1
Training an SAE from scratch requires substantial data and compute. The described analysis instead studies activations in MLP expansion layers, which contain more neurons. section 10.4 section 11.5.1
Steering with a feature
Clamping the feature associated with the Golden Gate Bridge led Claude to mention the bridge in virtually any response, regardless of the question. section 11.5.1

