A large language model (LLM) is a machine-learning model trained on enormous collections of human-authored text. It converts language into an internal numeric representation, then generates new text one token at a time. Its fluent output can support conversation, summarization, question answering, instruction following, writing, and code generation, but fluency does not guarantee truth, reasoning, or reliability. section 1.0 section 4.2
The name describes a family of systems rather than one product. ChatGPT is an LLM and a form of generative AI; GPT means Generative Pretrained Transformer. Other systems use related structures, licensing, and deployment choices, while commercial implementations may keep some details private. section 1.1 section 1.6
A simplified path from prompt to response looks like this:
- Split the input into tokens, which may be words, word fragments, punctuation, or other units rather than individual characters. section 2.0 section 2.3.1
- Convert tokens into vectors, add information about position, and use transformer layers to model relationships through attention. section 3.0 section 3.5
- Produce probabilities for possible next tokens, select one through sampling or unembedding, append it, and repeat the loop. section 3.5
- Learn during training by predicting missing next tokens in human text and reducing prediction error; this rewards text that resembles the data, not necessarily text that is factual. section 4.2.1 section 4.2
Why familiar tasks work better
A request to write Python code that calculates pi resembles abundant material in the training data, so the output may be useful. Translating that code into the obscure language Modula-3 requires more extrapolation and becomes less dependable. Similarly, unusual spelling and word-level puzzles can expose tokenization weaknesses. section 4.3 section 2.3.1
Figure 1.2 Generative AI takes some input (numbers, text, images) and produces a new output (usually text or images). Any combination of input or output options is possible, and the nature of the output depends on what the algorithm was trained for. It could be to add detail, rewrite something to be shorter, extrapolate missing portions, and more.
Use the input-output map to see how generative systems transform text, numbers, or images into new outputs for tasks such as rewriting, completion, and detail addition. section 1.1
LLMs are strongest when a task resembles patterns present in their training data and has a bounded purpose. They can be useful for low-effort coding, data processing, summarization, and idea generation, especially when inputs and prompts are constrained. Their performance can degrade as tasks demand novelty, intricate logic, or unsupported facts. Errors, stereotypes, bias, buggy code, privacy problems, and adversarial misuse make verification and careful scoping essential. section 4.3 section 4.4 section 1.8
