liveBook · Manning

SelfQueryRetriever

As explained in AI Agents and Applications

SelfQueryRetriever is a LangChain retriever that uses a language model to infer metadata constraints from a natural-language question before retrieving documents.

SelfQueryRetriever turns a natural-language question into two retrieval signals: a semantic query and metadata constraints. This lets a vector store narrow results by fields such as destination or region before applying semantic search. section 10.2

Prepare the collection by attaching metadata to each chunk, then describe the searchable fields with AttributeInfo. Build the retriever with an LLM, the vector collection, and those field definitions; invoke it with the user question. For “Tell me about events or festivals in the UK town of Newquay,” the retriever infers Newquay as the destination and returns matching chunks. section 10.2 section 10.2.2

Choose the retrieval approach according to how much control and setup the application needs:

  • Use an explicit metadata filter when the application already knows the constraint, such as destination = Newquay. section 10.2.2
  • Use SelfQueryRetriever when the question should determine the filter automatically without requiring the caller to supply one. section 10.2.2
  • Use a structured LLM function call when stronger typing and custom filter construction justify additional implementation. section 10.2.2
  • Budget for extra LLM cost and latency, because filter inference is performed by a language model. section 10.2.2
Figure 10.1 Self-querying workflow. The original question turns into a semantic query with an embedded metadata filter. When this enriched query runs on the vector store, it first selects chunks matching the metadata filter and then applies semantic search on this refined set.
figure

Follow the workflow from the original question to an enriched query, metadata selection, and semantic search over the reduced collection. section 10.2

How AI Agents and Applications covers it

How deep it goes
Explains metadata preparation, field descriptions, construction, invocation, alternatives, and cost, but not the internal inference prompt.
Examples
Uses UK tourism chunks tagged with destination and region, querying for events or festivals in Newquay.
What it assumes
Assumes a metadata-enriched vector collection and an available language model, while defining metadata fields from the implementation level.
What you can do after
Build filtered retrieval with LangChain by defining metadata fields, configuring SelfQueryRetriever, and invoking it with a question.
How it connects
Connects vector stores, metadata filters, semantic search, language models, and structured function calling.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Agents and Applications. The section links point to the parts of the book it draws on.