A searcher asking a question may want an answer, not a list of documents to read. Extractive question answering finds relevant words, sentences, or passages in search results and returns them directly. It searches and ranks answers within documents, making it more targeted than returning documents alone. section 1.3.5 section 14.0
Unlike abstractive question answering, extraction does not synthesize a new response: the answer is taken from the searched documents. Its usefulness therefore depends on finding context that contains an answer; the returned span is a likely answer, not a guarantee. section 1.3.5 section 14.1.1
A model can select a short answer span from a question and a retrieved context:
- Tokenize the question and context so the model can work with their constituent tokens. section 14.1.1
- Estimate the probability that each token starts the answer and the probability that each token ends it. These are two probability mass functions over discrete tokens. section 14.1.1
- Choose start and end positions and extract the corresponding tokens as the answer. The demonstrated implementation identifies the strongest normalized start and end positions. section 14.1.1
Finding a definition
For the question “What are minimalist shoes?”, a retrieved passage describes several features of the shoes. The model can select the shorter span “shoes intended to closely approximate barefoot running conditions” rather than returning the entire passage. section 14.1.1
Listing 14.1 Loading the tokenizer and model
from transformers import AutoTokenizer, AutoModelForQuestionAnswering model_name = "deepset/roberta-base-squad2" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForQuestionAnswering.from_pretrained(model_name)
Load a pretrained question-answering model and its tokenizer with the Transformers library before processing a question and context. section 14.1.1
Listing 14.3 Identifying an answer span from a tokenized context
start_tokens = []
end_tokens = []
terms = tokenizer.convert_ids_to_tokens(input_ids)
start_token_id = 0
end_token_id = len(terms)
for i, term in enumerate(terms):
start_tokens.append(stylize(term, [0, 127, 255], start_logits_norm[0][i]))
end_tokens.append(stylize(term, [255, 0, 255], end_logits_norm[0][i]))
if start_logits_norm[0][i] == 1.0:
start_token_id = i
if end_logits_norm[0][i] == 1.0:
end_token_id = i + 1
answer = terms[start_token_id:end_token_id]
display(HTML(f'<h3>{clean_token(" ".join(answer))}</h3>')) #1
display(HTML(f'<pre>{" ".join(start_tokens)}</pre>')) #2
display(HTML(f'<pre>{" ".join(end_tokens)}</pre>')) #3
Inspect the example Python code for identifying start and end tokens and extracting the answer span. section 14.1.1
