liveBook · Manning

Ambiguity

As explained in LLMs in Production

Semantic ambiguity is the condition in which an expression supports multiple possible meanings or interpretations.

A model trained to take fast-food orders needs more than examples of routine orders and cashier replies. Customers may make unusual requests, such as a burger with two top buns, and their wording may also need interpretation. A dataset should reflect the task the model will actually face. section 4.3.2

Semantic ambiguity matters when a request supports more than one interpretation. If training data contains too few examples of ambiguity being resolved during the target task, model performance may be unstable. Unusual requests are not necessarily ambiguous, but both expose the limits of a generic order dataset. section 4.3.2

When preparing language data:

  • Check that examples match the job the model must perform. section 4.3.2
  • Check for enough examples that resolve ambiguity while completing that job. section 4.3.2

How LLMs in Production covers it

How deep it goes
Explains the need for resolved examples and the risk of unstable performance, but gives no annotation procedure.
Examples
Uses fast-food orders, including a request for two top buns, to illustrate why routine examples may be insufficient.
What you can do after
Evaluate/understand it by checking whether a dataset includes enough task-relevant examples that resolve ambiguity.
How it connects
Connects semantic ambiguity to instruction-tuned models, dataset cleaning, speech acts, and task-specific training data.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

Compare how 4 books explain ambiguity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.