liveBook · Manning

Coverage gap

As explained in LLM Customization and Fine-Tuning

A shortfall in the range of cases or situations represented by examples or a dataset.

A coverage gap means the few-shot examples do not span enough cases. When coverage is the problem, expand to many-shot examples; move to fine-tuning if many-shot plateaus or its per-request cost is too high. section 4.9.2

Figure 4.7 The graduation decision tree: start with prompting, add RAG when the diagnostic shows a knowledge gap, and graduate to parameter-efficient fine-tuning when prompting plateaus or per-request cost flips the economics.

Follow the decision tree to distinguish a coverage gap, which calls for many-shot prompting, from a knowledge gap, which calls for RAG. section 4.9.2

How LLM Customization and Fine-Tuning covers it

How deep it goes
Defines the decision path and its escalation conditions, stopping short of detailing how to implement many-shot prompting.
Examples
Contrasts adding RAG for a knowledge gap with scaling to many-shot examples for a coverage gap.
What it assumes
Starts from a few-shot baseline and assumes a diagnosis of whether the shortfall concerns knowledge or coverage.
What you can do after
Apply the decision path to choose many-shot prompting when coverage is the diagnosed gap.
How it connects
Connects few-shot prompting and many-shot prompting with RAG and fine-tuning as alternative responses to different gaps.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.