Fine-tuning adapts a pretrained model by continuing parameter training on task-specific data. Pretraining supplies coherent language generation; fine-tuning aligns that capability with a causal Markov kernel. section 13.3.2
A practical workflow separates the modeling task into conditional text-generation problems:
- Fine-tune GPT-2 to generate action vignettes from the King’s marginal distribution, using an empty prompt. section 13.3.2
- Fine-tune BART sequence-to-sequence models to map King actions to Prince actions, and both actions to Kingdom fate. section 13.3.2
Causal vignette pipeline
A fine-tuned King model generates a war declaration; a Prince model generates abdication in response; a Kingdom model generates resource depletion and collapse. These models can then serve as distributions inside a Pyro causal model.
