A search result may contain the answer to a question without making the searcher open and read the whole document. Extractive question answering finds and returns the relevant words or passage from that document. Unlike abstractive question answering, it does not write a new answer by synthesizing information. section 1.3.5 section 15.2
For a question and a document likely to contain its answer, a model can identify an answer span in three steps:
- Break the document context into tokens. section 14.1.1
- Estimate, for each token, how likely it is to start or end the answer. section 14.1.1
- Use the predicted boundaries to extract a span from the original context. section 14.1.1
Answering a question from a search result
Given “What are minimalist shoes?” and a document containing the phrase “shoes intended to closely approximate barefoot running conditions,” the model can return that phrase as its answer. The phrase comes from the supplied document; it is not newly composed. section 14.1.1 section 15.2
Listing 14.1 Loading the tokenizer and model
from transformers import AutoTokenizer, AutoModelForQuestionAnswering model_name = "deepset/roberta-base-squad2" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForQuestionAnswering.from_pretrained(model_name)
The Python listing loads a pretrained question-answering tokenizer and model with Transformers. section 14.1.1
Listing 14.3 Identifying an answer span from a tokenized context
start_tokens = []
end_tokens = []
terms = tokenizer.convert_ids_to_tokens(input_ids)
start_token_id = 0
end_token_id = len(terms)
for i, term in enumerate(terms):
start_tokens.append(stylize(term, [0, 127, 255], start_logits_norm[0][i]))
end_tokens.append(stylize(term, [255, 0, 255], end_logits_norm[0][i]))
if start_logits_norm[0][i] == 1.0:
start_token_id = i
if end_logits_norm[0][i] == 1.0:
end_token_id = i + 1
answer = terms[start_token_id:end_token_id]
display(HTML(f'<h3>{clean_token(" ".join(answer))}</h3>')) #1
display(HTML(f'<pre>{" ".join(start_tokens)}</pre>')) #2
display(HTML(f'<pre>{" ".join(end_tokens)}</pre>')) #3
The listing shows how predicted start and end positions become a displayed answer span. section 14.1.1
This approach makes search more targeted: it can rank answers within results instead of returning only documents. Its boundary is also clear. If the needed answer is not present in the searched text, extraction cannot create it. section 1.3.5 section 15.2
