Natural language generation (NLG), or text generation, produces human-readable language from a computational process. It is the writing side of natural language processing: a chatbot uses it to compose a reply, while a search engine might use it to summarize results. Pasting a name into a fixed greeting is string manipulation; other generation methods commonly represent output numerically and decode it into readable text. section 1.1.2 section 1.8
A language model generates by estimating which word is likely to follow the words already present. Training on text lets it learn word-occurrence patterns; a prompt supplies the starting context. It chooses a word, adds that word to the context, and repeats. The most probable choice need not win every time: a temperature setting affects how often less-probable choices are selected. This is a way to produce new text, not a guarantee that the text is true. section 10.2 section 11.0
Generation serves different jobs depending on the surrounding pipeline:
- Autocomplete finishes a sentence; spelling and grammar tools help revise one. section 1.1.2
- Chatbots compose responses, while search engines can consolidate or summarize results. section 1.8
- Code assistants can suggest more than a retrieved snippet: a Copilot example starts with a TypeScript comment and function signature and generates the remaining function. section 1.3.1
Try a small next-word generator
Fit scikit-learn’s CountVectorizer to bigrams from sentences in 100 Wikipedia articles. Calculate how often each second word follows a given first word, then use Python’s random.choice to autocomplete sentences. Compare the results with an LLM such as Llama 2. The experiment makes next-word prediction tangible without treating the simpler model and an LLM as equivalent. section 10.4 section 10.2
Figure 10.2 Stochastic chameleons deciding words one at a time
The word-by-word probability diagram shows one possible path through generation; each newly chosen word affects the context for later choices. section 10.2
Where factual accuracy matters, distinguish fluent wording from justified claims. A knowledge graph can supply stored facts for grounded text generation and support logical inferences; next-word statistics alone do not provide that grounding. Generated content can also contain mistakes or bias, so useful output still needs scrutiny. section 11.0 section 1.3
