GPT-2 is a causal transformer language model for generating text from a prompt. It predicts a next token from the preceding tokens, then uses generated tokens as context for subsequent predictions. This makes it useful for experimenting with text generation, but a fluent continuation is not necessarily correct or useful. section 10.2 section 10.2.1 section 10.3.11
A practical generation workflow:
- Load matching pretrained model and tokenizer weights with Hugging Face: GPT2LMHeadModel.from_pretrained('gpt2') and GPT2Tokenizer.from_pretrained('gpt2'). Encode the prompt with that tokenizer and decode the generated token IDs to read the result. section 10.2.1
- Inspect predictions before generating a passage: for the prompt “NLP is a”, take the last position’s logits, apply softmax, and rank candidate tokens. In the shown run, the top decoded candidates are “ new” and “ non”. section 10.2.1
- Change the generation strategy when results are repetitive. One demonstrated setting uses nucleus sampling with do_sample=True, top_p=0.92, and max_length=50; it changes the continuation, not its guarantee of accuracy. section 10.2.1
From an awkward continuation to a domain-tuned model
With the prompt “NLP is”, pretrained GPT-2 produces an incorrect description of NLP and repeats a database phrase. To adapt its generation patterns, filter a corpus to natural-language lines, tokenize and pad each line as a training example, split examples into training and evaluation sets, and fine-tune a fresh pretrained GPT-2 with a language-modeling data collator and Hugging Face Trainer. Compare continuations from the original and fine-tuned models rather than assuming fine-tuning makes every answer reliable. section 10.2.1 section 10.2.2
GPT-2 supplies generation rather than retrieval. For question answering grounded in supplied documents, pair semantic search with a generative model and instruct that model not to invent unsupported answers. The demonstrated retrieval-augmented pipeline uses FLAN-T5, not GPT-2; GPT-2 is cited there as an example of a causal generator. section 10.3.11
