A foundational model is a pretrained language model that supplies broad language understanding, reasoning capability, and general knowledge before adaptation to a particular task or domain. Pretraining builds this base by learning to predict tokens from massive raw-text corpora; later instruction tuning, reinforcement learning, or fine-tuning directs its behavior toward useful tasks such as classification, summarization, and generation. section 1.1 section 1.4.1
Figure 1.1 Training process for general-purpose LLMs. Phase 1 (pre-training) ingests massive raw text corpora to build foundational language understanding; Phase 2 (instruction tuning/RL) uses curated labeled datasets to align the model toward helpful, accurate behavior.

The training pipeline separates broad pretraining from the later alignment stage that teaches more complex task behavior. section 1.1
A foundation is valuable because specialization can reuse knowledge that would be costly to recreate from scratch. Fine-tuning supplies curated examples for a target behavior and adjusts the model toward domain-specific outputs. Its success depends on the starting model: fine-tuning can refine and align existing knowledge, but limited domain grounding makes specialization inefficient and can yield unstable results. section 1.4.1 section 2.3.3
Select a foundational model by weighing the starting point against the operating constraints:
- Match baseline task capabilities to the work, such as reading comprehension, salient-information selection, and coherent generation for summarization. section 2.5.1
- Check domain exposure to relevant terminology, standards, reasoning patterns, and task structures; closer exposure reduces required training data, time, and compute. section 2.3.1 section 2.3.3
- Fit model size and deployment requirements to available compute and infrastructure; larger models may offer stronger capabilities, while smaller models can suffice for well-defined tasks. section 1.4.2
- Account for privacy, security, and regulatory constraints when sensitive financial, legal, or health data cannot be sent to third-party cloud services. section 1.1.2
- Verify that a public evaluation dataset was not already present in the model’s training data, or fine-tuning results may be misleading. section 2.5.1
Financial analyst-report summarization
A general web-trained model may summarize an analyst report yet miss which findings are material or how to contextualize earnings guidance. Qwen2.5-7B-Instruct is selected as a starting point because of reported exposure to financial reports, filings, and market commentary, plus summarization ability and practical 7B-scale infrastructure requirements. Fine-tuning can then sharpen that existing foundation for the target workflow. section 2.5.1
General-purpose foundations trade breadth for specialization. They can provide adequate results across many domains, but specialized models can outperform them on focused financial, medical, or legal work. Frontier-scale development and local deployment also impose high cost and infrastructure burdens, making adaptation of an existing model a practical alternative to training a new one. section 1.1.2 section 1.4.1
Figure 2.2 LoRA adapter swapping for different domains or use cases. A single frozen foundation model serves as the shared backbone for multiple lightweight domain adapters. Each adapter adds a small set of trainable weights alongside the frozen layers — less than 0.1% of the base model's parameters — and can be swapped in or out without modifying the underlying model.

Parameter-efficient fine-tuning preserves one frozen foundation model while small swappable adapters provide distinct domain behaviors. section 2.2
Parameter-efficient fine-tuning extends the foundation-model approach by keeping base weights frozen and training small adapters instead. A single model can support separate finance and customer-support adaptations, reducing checkpoint storage and enabling experimentation. LoRA follows this pattern by learning low-rank updates in targeted layers rather than modifying the pretrained weights. section 2.2 section 3.5.1





























