liveBook · Manning

Grounded language model

As explained in Natural Language Processing in Action, Second Edition

A grounded language model is a language model whose language behavior is linked to physical experience or verified external knowledge.

A grounded language model links language behavior to physical experience or verified knowledge. Sensors such as cameras and microphones let an embodied system compare language-based assumptions with the surrounding world. section 10.2.3

Grounding addresses a central weakness of text-only language models: fluent output can still contain nonsense because internet text provides no direct opportunity to detect mistakes against reality. Embodied feedback can expose faulty common-sense rules and support correction. section 10.2.3

Counting apples

One apple plus another apple produces two apples in the physical world. A grounded model could use that observable result to check its language-based reasoning and improve counting or addition. section 10.2.3

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Defines grounding through sensors, embodiment, and error correction, but stops short of implementation details or a training procedure.
Examples
Uses an embodied robot, camera and microphone sensing, walking and talking, and apple addition as grounding examples.
What it assumes
Assumes familiarity with language models and introduces embodiment, sensing, and reality-based correction through examples.
What you can do after
Evaluate how physical feedback could expose language-model errors and improve common-sense reasoning.
How it connects
Connects language models with embodied AI, sensors, common sense, reasoning, hallucination, and the physical world.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.