Context is the surrounding information and circumstances that shape how language or a task is interpreted. Words rarely carry meaning on their own: intent and the surrounding conversation also matter. For language models, the information available during generation helps define the task and frame the response. section 2.0 section 6.4.1
An answer draws on two sources: patterns encoded in the model’s parameters during training and text supplied at inference time. The prompt provides the active context; the parameters shape how the model transforms that input and predicts continuations. Encoded knowledge is not retrieved as if from a database. section 6.4.1
This distinction sets a practical boundary. Attention can connect only tokens in the active sequence. If a fact is neither encoded in the parameters nor supplied in the prompt, it cannot influence the answer. Documents, post-training events and earlier exchanges are inaccessible unless their relevant information is provided; conversational continuity depends on the surrounding system reintroducing prior turns. section 6.4.1
Available context does not guarantee a reliable answer. Instructions, examples, evidence and prior turns can reinforce or displace one another. Next-token prediction favors plausible continuations without requiring factual verification, so a fluent response can extend weak patterns rather than acknowledge missing evidence. section 6.0 section 6.1.2
Context can also be built during a response. In chain-of-thought, intermediate steps become input for later tokens and may help guide a problem toward a valid answer. Such steps can serve as a scaffold or an explanation, but they can also be a misleading trace; their presence does not establish exactly how the model reached its answer. section 9.2.3





























