A large language model (LLM) is the text-generating core of a generative AI application: it receives natural-language text and produces text in return. Unlike conventional code that follows fixed steps, it works probabilistically, so the same carefully designed prompt cannot guarantee an identical output every time. section 1.2 section 1.3
A bare LLM has important operating characteristics:
- It is nondeterministic: outputs arise from probabilities rather than fixed logical rules. section 1.3
- It is stateless: each call starts clean unless an application supplies prior context. section 1.3 section 1.4
- It is pretrained: its built-in knowledge is fixed after training and does not update through ordinary use. section 1.3
- It is not explanatory: a response does not reveal why the model produced that particular result. section 1.3
Under the hood, an LLM learns patterns through machine-learning training. GPT-style models represent tokens as numerical embeddings, use context to estimate the most likely next token, and repeat that completion process to form text. Chat-oriented systems can add supervised and reinforcement-learning refinement after initial pretraining. section 1.5 section 1.6 section 1.7
Why a chat can remember a name
After a user says, “Hi, I’m Andrea,” a separate request asking “What’s my name?” has no answer by itself. A chat application can create the appearance of memory by sending relevant earlier conversation history together with the new request; the LLM itself still begins each call without retained state. section 1.4
Useful applications surround the model with components that compensate for these limits. Retrieval-augmented generation can provide current or organization-specific documents; agents and tools can fetch live information, calculate precisely, or trigger external actions. The LLM remains the language and reasoning component rather than the system that inherently stores memory, knows current facts, or performs actions. section 1.7 section 4.0 section 5.1
Figure 1.11 How a GPT architecture generates sentences
Follow the high-level GPT architecture from prompt through token-by-token sentence generation. section 1.6
