A pretrained model is an existing, general-purpose model used as-is or with limited adaptation instead of rebuilt from scratch. Off-the-shelf models are the practical version of this idea: ready to use, easy to deploy, and usually customized first through prompts. section 3.15 section 3.3
Clarifying an ambiguous request
Suppose you ask an existing language model to make a table of 200-meter world records. The request leaves the format and contents unclear. Adding a few demonstrations in the prompt—few-shot prompting—steers the model without changing its internal parameters. section 3.3
A sensible adaptation ladder is:
- Start with the pretrained model and improve the prompt, including examples when the task is ambiguous. section 3.3
- Add retrieval-augmented generation when the model needs substantial contextual information. section 3.3
- Fine-tune only when prompting and added context are insufficient, or when a smaller specialized model is preferable. section 3.15 section 3.3
Fine-tuning creates an altered copy by continuing training on task-specific data and adjusting internal parameters. That is more involved than using the pretrained model directly, which remains the default starting point for many customizations. section 3.3 section 3.15
