Contextualized late interaction is a retrieval technique for RAG that compares context-aware representations of individual document and query tokens instead of compressing each item into one vector. section 15.2.1
The document-side process generates an embedding for each token, while each embedding incorporates that token’s context within the document or a long surrounding span. This preserves keyword-level content context without requiring unlimited overlapping chunks. section 15.2.1 section 15.2.1
Search pipeline implication
A retrieval system can store multiple contextual vectors per document and compare them with query-token representations. Multivalued vector fields in Vespa or Qdrant support this representation, while ColBERT-family models provide prominent examples. section 15.2.1 section 15.2.1
