ColPali represents rendered document pages visually for late-interaction retrieval, avoiding dependence on imperfect text extraction. It targets pages where layout, figures, or rendering artifacts carry meaning. section 15.6.5
Choose it with production tradeoffs in view:
- Compare it with text retrieval rather than assuming visual retrieval always wins: the cited ViDoRe result gives ColPali 81.3 nDCG@5 versus 67.0 for the best text pipeline. section 15.6.5
- Budget storage before indexing: late interaction stores roughly a thousand vectors per page, potentially turning million-page indexes from gigabytes into terabytes. section 15.6.5
- Consider hybrid retrieval: combining visual top pages with text top passages can outperform choosing one representation, while reranking results vary by dataset. section 15.6.5
