A customer ordering a meal might make a routine request—or ask for a burger with two top buns. Both are orders, but a dataset of only standard exchanges may leave a model unprepared for the second. The words in a dataset need to reflect what people will actually try to do with the model. section 4.3.2
A speech act is an utterance viewed as a communicative action, such as a request, assertion, or instruction. For an LLM task, the practical question is whether training examples contain the kinds of actions and ambiguities the model must handle. If they do not, performance can be unstable even when the dataset appears relevant. section 4.3.2
When preparing language data:
- Start with what the model must do, then check whether the dataset includes enough examples of those interactions. section 4.3.2
- Check for examples that resolve semantic ambiguity, not just straightforward prompts and responses. section 4.3.2
- Use annotation to analyze the data, and aim for a distribution of speech acts appropriate to the task when finetuning. section 4.6
