A chatbot can answer a question, suggest a sentence ending, or summarize a page. Behind these tasks is a language model: a system that learns patterns in language and uses them to predict or generate text. A large language model (LLM) is trained on vast amounts of data, allowing one model to work across many tasks rather than only one. section 1.5 section 1.3
The basic idea is to predict a missing or next piece of text from its context. These pieces, called tokens, can be words or parts of words. In self-supervised training, the text supplies its own prediction task, so people need not label every example. The resulting model can then be adapted for particular abilities, such as following instructions or writing code. This describes the process at a high level, not the exact path from a prompt to a reliable answer. section 1.2 section 1.3 section 2.16
Using context to complete a sentence
After a paragraph about someone learning to take a photograph, a model is asked to complete: “He nodded sheepishly, threw his cigarette away and took the _____.” The intended word is “camera.” The example tests whether the model can use earlier context, not merely the nearest words. section 1.5
Language models can support different kinds of work:
- Generate text for chatbots or autocomplete, and adapt to formats such as posts or code. section 1.5
- Answer questions, classify text, translate languages, or produce summaries. section 1.4 section 1.10
- Assist programmers by suggesting code, while leaving security, licensing, and review as concerns. section 1.7
Modern LLMs grew from advances in natural language processing, especially attention and transformer models, alongside more available web text and computing power. Attention helps a model weigh relevant parts of a sequence. Large-scale pretraining can give a model broad abilities, but a fluent response is not proof that it is correct or that the model consistently reasons through a problem. section 1.2 section 1.23 section 1.9
Scale also brings costs and risks. Web training data can carry cultural bias, objectionable content, and sensitive information into a model; outputs can be made up or unsafe. Training and use require substantial computing resources, with environmental effects that are difficult to measure precisely. Treat a generated answer as something to check, especially when accuracy, privacy, or safety matters. section 3.20 section 1.22 section 1.14
