14 Question answering with a fine-tuned large language model

 

This chapter covers

  • Building a question-answering application using an LLM
  • Curating a question-answering dataset for training
  • Fine-tuning a Transformer-based LLM
  • Integrating a deep-learning-based NLP pipeline to extract and rank answers from search results

We covered the basics of semantic search using Transformers in chapter 13, so we’re now ready to attempt one of the hardest problems in search: question answering.

Question answering is the process of returning an answer for a searcher’s query, rather than just a list of search results. There are two types of question-answering approaches: extractive and abstractive. Extractive question answering is the process of finding exact answers to questions from your documents. It returns snippets of your documents containing the likely answer to the user’s question so they don’t need to sift through search results. In contrast, abstractive question answering is the process of generating responses to a user’s question either as a summary of multiple documents or directly from an LLM with no source documents. In this chapter, we’ll focus primarily on extractive question answering, saving abstractive question answering for chapter 15.

By solving the question-answering problem, you will accomplish three things:

14.1 Question-answering overview

14.1.1 How a question-answering model works

14.1.2 The retriever-reader pattern

14.2 Constructing a question-answering training dataset

14.2.1 Gathering and cleaning a question-answering dataset

14.2.2 Creating the silver set: Automatically labeling data from a pretrained model

14.2.3 Human-in-the-loop training: Manually correcting the silver set to produce a golden set

14.2.4 Formatting the golden set for training, testing, and validation

14.3 Fine-tuning the question-answering model

14.3.1 Tokenizing and shaping our labeled data

14.3.2 Configuring the model trainer

14.3.3 Performing training and evaluating loss

14.3.4 Holdout validation and confirmation

14.4 Building the reader with the new fine-tuned model

14.5 Incorporating the retriever: Using the question-answering model with the search engine