14 Question answering with a fine-tuned large language model

With the basics of semantic search using transformers well understood from Chapter 13, we’re now ready to attempt one of the hardest problems in search: Question Answering.

Question Answering is the process of returning an answer for a searcher’s query, rather than just a list of search results. There are two types of question-answering approaches: extractive and abstractive. Extractive question answering is the process of finding exact answers to questions from your documents. It returns snippets from your documents containing the likely answer to the user’s question to prevent them from needing to sift through search results. In contrast, Abstractive question answering is the process of generating responses to a user’s question either as a summary of multiple documents or directly from a Large Language Model with no source documents. In this chapter, we’ll focus primarily on extractive question answering.

14.1 Question answering overview

14.1.1 How a question-answering model works

14.1.2 The retriever-reader pattern

14.2 Constructing a question-answering training dataset

14.2.1 Using guesses from the pre-trained model for human-in-the-loop labeling

14.2.2 Converting the labeled data into the SQuAD data format

14.3 Fine-tuning the question-answering model

14.3.1 Tokenizing and shaping our labeled data

14.3.2 Configuring the RobertaForQuestionAnswering trainer

14.3.3 Performing training and evaluating loss

14.3.4 Hold-out validation and confirmation

14.4 Building the reader with the new fine-tuned model

14.5 Incorporating the retriever: using the Question-Answering model with the search engine

14.5.1 Step 1: Querying the Retriever

14.5.2 Step 2: Inferring answers from the Reader model

14.5.3 Step 3: Re-ranking the answers

14.5.4 Step 4: Returning results by combining the Retriever, Reader, and Re-ranker

14.6 Summary