A searcher asking “What are minimalist shoes?” may want a short answer, not a list of pages to inspect. Question answering returns an answer to a natural-language query. Extractive question answering selects an answer from available documents; abstractive question answering generates a response, potentially by summarizing documents or without using source documents. The pipeline below focuses on extraction. section 14.0 section 14.1.2
An extractive system can separate retrieval from reading. A search engine retrieves a small set of likely documents, then a reader pairs each document’s text with the question and predicts an answer. Keeping only three to five candidates makes reading practical, but raises the stakes for retrieval: if the relevant text misses that window, the reader cannot extract it. The retriever might use BM25 or an embedding index. section 14.1.2
Figure 14.4 Retriever-reader pattern for extractive question answering
Follow the question from document retrieval through answer extraction and reranking; each stage has a distinct job. section 14.1.2
A practical build has four stages:
- Prepare question–context pairs. Short question-and-answer posts, such as those in an outdoors forum, are easier to adapt than long documents; lengthy text may need paragraph splitting and manually devised questions. section 14.2
- Run a pretrained question-answering model over the pairs to make a silver set of proposed answer spans. Check and correct those guesses manually to create a more trustworthy golden set, then split it for training, testing, and validation. section 14.2 section 14.2.2
- Fine-tune the reader on the corrected data and check it against held-out questions and contexts. A reported holdout check found exact matching answers for 7 of 10 examples—not perfect accuracy. section 14.1.2 section 14.4
- At query time, retrieve top-K documents, predict an answer and score for each question–context pair, and rerank the predictions by descending reader score. section 14.5
From a retrieved post to an answer
For “What are minimalist shoes?”, a retrieved post contains the phrase “shoes intended to closely approximate barefoot running conditions.” The extractive reader can return that phrase rather than requiring the searcher to read the whole post. Predictions also carry scores and start/end positions; treat them as candidates to check, not as proof that an answer is correct. section 14.1.2 section 14.2.2
Good answers depend on both relevant retrieved contexts and reliable labels for the reader. Model-generated silver labels contain mistakes, while the accuracy and breadth of the corrected golden set limit the trained model. Question-answering systems can also use rules and ontologies instead of machine learning; an LLM reader is one implementation choice. section 14.1.2 section 14.2 section 1.1
