A foundational model is trained on large volumes of data to learn broadly useful patterns, rather than being built for one narrowly defined task. Its learned representations can be reused across downstream tasks. Large language models (LLMs) are foundational models adapted for language work; the broader category also applies to unstructured data such as images and audio. section 2.1 section 2.2
Pretraining produces the starting model. In language applications, a base LLM learns patterns from a large text corpus and can generate text before task-specific training. It is versatile, but that does not mean it reliably follows a user's instructions or meets a particular domain's accuracy requirements. A base LLM is one kind of foundational model, not a synonym for every foundational model. section 2.5 section 9.5.1
Choose how to make the starting model useful for the job:
- Prompt engineering supplies cues and explicit directions without changing the model's learned capabilities. section 6.1.1 section 2.5
- Fine-tuning trains a pretrained model further on task-specific data. Select a suitable model, curate data, train, evaluate on separate data, and then deploy if performance is acceptable. It costs additional resources and can overfit or erode previously learned capabilities. section 9.2.1 section 2.8.8
Starting an enterprise assistant
A model such as GPT-4 does not become a useful assistant merely by being available. Start with a small pilot tied to a business process, such as drafting reports. Define the use case, connect the model to the workflow, and test whether its output meets the task's requirements before expanding deployment. For model selection, compare performance and risks across relevant scenarios; HELM includes measures such as accuracy, robustness, bias, toxicity, and efficiency. section 1.7 section 12.3.3
In an application architecture, foundational LLMs sit in the model layer alongside fine-tuned and small language models. Pick among them for the task, available data, and resources—not simply for model size. Establish governance for sensitive or harmful output as part of deployment. section 10.5 section 1.7
