A multimodal embedder is a model that maps inputs from multiple modalities, such as text and images, into vector representations.
A multimodal embedding model maps inputs such as text and images into vector representations. For document retrieval, a single-vector embedder can turn a page screenshot into an ordinary vector that an existing vector store can index. section 15.6.5
Use the simplest visual strategy that addresses the failure mode:
Keep a text pipeline with a reranker when extraction works reliably; attach the page image to the generation context when an answer lives in a figure, table, or scan. section 15.6.5
Choose a visual index when parsing fails across the corpus rather than on isolated pages; a single-vector multimodal embedder avoids changing the existing indexing interface. section 15.6.5
Consider Cohere Embed 4 or Voyage voyage-multimodal-3 when a commercially available screenshot-to-vector path is required. section 15.6.5
How Retrieval Augmented Generation, The Foundational Ideas covers it
How deep it goes
Defines screenshot-to-vector indexing and selection tradeoffs, but stops short of model-training or embedding-internals explanations.
Examples
Uses page screenshots and names Cohere Embed 4 and Voyage voyage-multimodal-3 as single-vector embedding options.
What it assumes
Assumes familiarity with text pipelines, rerankers, vector stores, and document parsing.
What you can do after
Evaluate and select between text retrieval, generation-time images, and visual indexing for document question answering.
How it connects
Connects multimodal embedding models to vector stores, rerankers, visual indexes, and late-interaction retrieval.
embedding modelkind of · An embedding model maps inputs to numerical vectors that represent salient features or relationships in a learned or defined space.
vector spaceuses · A vector space is a set equipped with vector addition and scalar multiplication over a field that satisfy the vector-space axioms.
visual retrievalalternative to · Visual retrieval finds relevant document pages by matching visual representations rather than relying solely on extracted text.
encodercontrasts with · An encoder is a component or process that transforms input into an encoded or internal representation for subsequent processing, transmission, or decoding.
This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.