liveBook · Manning

Coreference resolution

As explained in Rearchitecting LLMs

Coreference resolution is the task of determining which previously mentioned entity a referring expression, such as a pronoun, denotes.

Coreference resolution determines which entity a referring expression, such as a pronoun, denotes. WinoGrande tests this by asking a model to choose which of two entities fills a sentence’s blank. section 4.3.3

For depth-pruning analysis, WinoGrande examples are formatted with the sentence, candidate entities, and correct answer. Hooks capture each model block’s inputs and outputs; cosine similarity between them contributes to a block-importance score. section 4.3.3

WinoGrande sentence

“Ian volunteered to eat Dennis’s menudo after already having a bowl because _ despised eating intestine.” The options are Ian and Dennis; the answer is Dennis. section 4.3.3

Figure 4.5 In the block evaluation pipeline, hooks are registered, a forward pass of the model is executed using the dataset, and the activations are captured to calculate the block importance. The result is a list containing the block identifier with the assigned importance.

Shows how registered hooks, a model forward pass, and captured activations produce block-importance scores. section 4.3.3

How Rearchitecting LLMs covers it

How deep it goes
Details dataset formatting, forward hooks, activation capture, and cosine-based block scoring.
Examples
Uses a sentence about Ian, Dennis, and menudo to illustrate choosing the entity referred to by a pronoun.
What you can do after
Evaluate model-block importance using activations captured on formatted WinoGrande examples.
How it connects
Connects pronoun resolution with WinoGrande and neural-network depth pruning.

Also explained in

Compare how 4 books explain coreference resolution →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.