A large language model (LLM) is a broad-purpose neural model trained on enormous token collections to predict language and support tasks such as classification and summarization. Its scale enables versatile behavior, including few-shot learning, but can create limitations for specialized applications. section 1.1 section 1.1.1
Core characteristics include:
- Stores learned knowledge in billions or trillions of neural-network parameters represented by numerical weights. section 1.1.1
- Processes text as tokens, typically subword units, and learns statistical relationships across very large corpora. section 1.1.1
- Builds foundational language understanding during pre-training, then improves task behavior through instruction tuning or reinforcement learning. section 1.1
- Provides a foundation for adaptation methods ranging from supervised and full-parameter fine-tuning to parameter-efficient approaches such as LoRA and QLoRA. section 3.8
Figure 1.1 Training process for general-purpose LLMs. Phase 1 (pre-training) ingests massive raw text corpora to build foundational language understanding; Phase 2 (instruction tuning/RL) uses curated labeled datasets to align the model toward helpful, accurate behavior.

Follow the training pipeline from raw-text pre-training to instruction tuning and reinforcement learning. section 1.1
Scale and adaptation
GPT-4 is estimated to contain more than one trillion total parameters, while Meta’s LLaMA 2 70B required approximately 1.7 million GPU-hours to train. These models can later be adapted with methods such as LoRA, which trains fewer than 1% of the parameters. section 1.1.1 section 3.8
























































































