Domain knowledge is information an agent needs for a particular subject or task that it cannot rely on its model alone to supply. Product documents, policies, and engineering records can provide that context. Unlike conversational memory, which records past interactions and experiences, knowledge usually comes from external documents or structured data. section 1.3.4 section 6.6
Keeping knowledge outside the prompt lets a collection grow independently of the model’s context window. Retrieval searches that collection for a question; augmentation puts selected results into the current prompt. Only the most relevant pieces fit in any one call, so selection matters as much as storage. section 6.1 section 1.3.4
Table 6.1 Sources, storage formats, and retrieval mechanisms of knowledge and memory used in agent applications
| Source |
Storage format |
Retrieval mechanism |
| Memory (short-term)—previous conversation thread |
Agent/user conversation thread stored as text |
Provided automatically as message history |
| Memory (long-term)—previous user conversations |
Summarized memories stored as text or in a vector database (semantic) |
Semantic/vector search |
| Knowledge (unstructured), documents (PDFs) |
Vector database (semantic) and text document index |
Semantic/vector search and keyword search |
| Knowledge (structured), database tables, structured logs |
Same format as source |
Relational database queries, log parsing |
Compare the routes from source to answer: documents can use semantic or keyword search, while database tables can be queried in their structured form. section 6.1
Choose retrieval to match the question:
- Use semantic search to find related passages, but add keyword search when exact terms, jargon, or numbers matter. section 6.6
- Use relational or graph queries when the answer depends on structured records or relationships. section 6.6
- Keep results current and scoped to the requester; filter by metadata such as product, version, date, role, or tenant. section 11.1.4
A filtered knowledge lookup
Suppose an agent needs to answer a question about a product policy. A retrieval tool can search a vector knowledge base and return five snippets filtered by product, version, date, role, and tenant. The agent should answer from those snippets, cite the retrieved passages, and say it does not know when they contain no answer. This keeps an unrelated or restricted document from becoming answer context. section 11.1.4 section 6.6
