Knowledge retrieval searches private or public sources for information relevant to a prompt, then supplies that context to an LLM. In retrieval-augmented generation (RAG), retrieval precedes natural-language generation and can support source checking for transparency and accuracy. section 2.2
Operationally, retrieval involves:
- Searching a corpus using embeddings and vector similarity. section 13.3
- Storing embeddings and metadata in a vector database, with approximate-nearest-neighbor indexes improving search speed. section 13.3
- Comparing RAG with prompt engineering or fine-tuning according to freshness, domain specificity, cost, and task requirements. section 2.2 section 2.6
Searching a local LanceDB table
Encode the query "blocked KV-cache," search the embedding column, and return the top three results. The returned passage discusses blocked KV-cache and continuous batching, with a similarity score of 0.7036938667297363. section 13.3
