liveBook · Manning

Relevance

Also called relevancy

Relevance is the degree to which information, a result, or a response pertains to a query, task, purpose, or information need.

How 17 Manning books explain it

Elasticsearch in Action, Second Edition

Relevance is the degree to which a search result matches a query's intent or information need.

How it is used
Ranks full-text search results so documents most appropriate to a user query appear first.
How deep it goes
Defines scored ranking, explains BM25 factors and field-length normalization, and defers detailed alternatives to similarity algorithms.
Examples
Uses Java book-title searches, Stack Overflow results, retail searches, and a Java-score result figure.
What you can do after
Apply it via Elasticsearch full-text queries to retrieve and rank documents for text searches.

Read how Elasticsearch in Action, Second Edition explains relevance →

LLM Evaluation and Alignment, The Foundational Ideas

Answer relevance is a generation-quality metric measuring how directly and appropriately a generated answer addresses the question or information need.

How it is used
Evaluates whether a RAG response addresses the user’s question, including in reference-free production monitoring.
How deep it goes
Defines a three-step back-translation procedure using generated questions, embeddings, and average cosine similarity, with a worked partial-answer example.
Examples
Uses a financial research question about Q3 revenue drivers and Q4 risks to illustrate an accurate but incomplete answer.
What you can do after
Diagnose RAG response failures by comparing answer relevance with faithfulness and retrieval scores.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains relevance →

MongoDB 8.0 in Action, Third Edition

A relevance score is a numeric value representing how closely a search result matches the search criteria.

How it is used
Ranks Atlas Search matches by how closely documents satisfy search criteria.
How deep it goes
Explains scoring factors and demonstrates score extraction, sorting, and summary statistics without deriving Lucene’s scoring formula.
Examples
Uses inspection searches with required text matches and a local fuzzy search that returns the three highest-scoring results.
What you can do after
Apply it via Atlas Search to display scores and order matching documents by relevance.

Read how MongoDB 8.0 in Action, Third Edition explains relevance →

Retrieval Augmented Generation, The Foundational Ideas

Answer relevance is a metric measuring how well a generated answer addresses the user's question.

How it is used
Diagnoses whether a RAG response answers the user's question, complementing retrieval-focused context relevance and grounding-focused faithfulness.
How deep it goes
Explains reverse-question generation, embedding similarity, and averaged scores, then works a numerical example without deriving embedding internals.
Examples
Works a mortgage reapplication query with similarities of 0.89, 0.82, and 0.91, yielding an answer-relevance score of 0.87.
What you can do after
Apply it via RAGAS to flag off-topic or wrongly specific answers, while checking faithfulness before trusting a relevant response.

Read how Retrieval Augmented Generation, The Foundational Ideas explains relevance →

AI-Powered Search

A relevance grade is an ordinal, categorical, or numeric assessment of how relevant a document is to a query.

How it is used
Uses query–document relevance grades as training labels and relevance scores to order search results.
How deep it goes
Explains manual binary judgments and click-derived probability grades, including prior adjustment for limited evidence.
Examples
Uses movie-query judgments and blue-ray click and examination counts to illustrate binary and probability grades.
What you can do after
Apply it via Python judgment and training-data utilities to label query–document pairs and generate click-derived grades.

Read how AI-Powered Search explains relevance →

AI for Everyday IT

Relevance is the quality of directly relating information or requirements to the task or use case being addressed.

How it is used
Guides AI prompts toward the information needed for a particular task or use case.
How deep it goes
Explains relevance through comparisons between broad requests and prompts with task-specific context and requirements.
Examples
Uses database selection, a small IT project, and job interview practice to illustrate relevance.
What you can do after
Apply it via an AI assistant by adding task context and requirements to prompts.

Read how AI for Everyday IT explains relevance →

AI Model Evaluation

The degree to which retrieved, recommended, or generated results address the intended query, context, task, or user need.

How it is used
Uses relevance to evaluate ranked recommendations and generated answers against user interests or queries.
How deep it goes
Works through offline recommendation evaluation and a structured judge rubric, while cautioning that benchmarks may miss live product behavior.
Examples
Uses a genre-based movie recommender, interaction-based proxy labels, Precision@K, and a JSON-scoring judge prompt.
What you can do after
Evaluate recommendation relevance with offline ranking metrics and structured judge rubrics while recognizing proxy-label and benchmark limitations.

Read how AI Model Evaluation explains relevance →

Build an Advanced RAG Application (From Scratch)

Answer relevancy measures how closely a generated answer addresses the original question, including through similarity to questions generated from the answer.

How it is used
Evaluates whether generated answers directly address user questions within a RAG generation pipeline.
How deep it goes
Defines the metric and its artificial-question similarity procedure, but does not provide implementation code or scoring details.
Examples
Generates artificial questions from an answer and compares them with the original question rather than relying on keyword matching.
What you can do after
Evaluate RAG answer targeting and compare system configurations using answer relevancy within the RAGAS metric suite.

Read how Build an Advanced RAG Application (From Scratch) explains relevance →

Effective Conversational AI

Answer relevance is the degree to which an answer directly addresses the user's question and context.

How it is used
Assesses whether a generated chatbot answer addresses the user’s particular question and context.
How deep it goes
Defines question–answer fit and contrasts it with faithfulness, but gives no scoring formula for answer relevancy.
Examples
Contrasts an order-status reply with unnecessary detail and generic restaurant advice with location-specific guidance.
What you can do after
Evaluate/understand whether a chatbot’s response answers the user’s actual question, separately from whether retrieved information supports it.

Read how Effective Conversational AI explains relevance →

Generative AI in Action

Answer relevancy is a metric that measures how directly a generated answer addresses the question or information need.

How it is used
Uses relevance to assess whether LLM outputs satisfy the context and intent of their intended application.
How deep it goes
Defines a five-point rubric and comparison steps, but defers deeper metric mechanics and full framework implementation.
Examples
Compares two summaries of a Gates Foundation article and demonstrates G-Eval and DeepEval evaluation workflows.
What you can do after
Evaluate summaries and other LLM outputs with explicit criteria, scores, comparisons, and ongoing test monitoring.

Read how Generative AI in Action explains relevance →

The Art of AI Product Development

A metric measuring how directly a generated answer addresses the user's query or intent.

How it is used
Evaluates whether generated RAG responses satisfy the user’s query, requested content, and required depth.
How deep it goes
Defines relevance, contrasts it with groundedness, and illustrates evaluation criteria without specifying a formal scoring method.
Examples
Uses an AI-trends report, requested depth, user-specific transformations, and style or tone feedback as evaluation cases.
What you can do after
Evaluate/understand RAG responses with query-specific criteria and identify when prompting, retrieval, or generation needs improvement.

Read how The Art of AI Product Development explains relevance →

WordPress Power Toolkit: Harness AI to build next-level websites

Search relevance is the degree to which a search result corresponds to the meaning and subject matter of a user's query.

How it is used
Explains how search systems prioritize meaningful results and how website elements support a page’s central message.
How deep it goes
Defines relevance, connects it to semantics and latent semantic indexing, and applies it to a visual critique without developing the underlying mathematics.
Examples
Uses query interpretation, the meanings of “right,” semantically rich content, and a disconnected potato image on an Instant Pot homepage.
What you can do after
Evaluate search content and website elements for alignment with user intent, subject matter, message, placement, and context.

Read how WordPress Power Toolkit: Harness AI to build next-level websites explains relevance →

Designing AI Systems

A retrieval relevance distribution is the statistical distribution of relevance scores produced for retrieved results across queries.

How it is used
Monitors retrieval quality over time through aggregate metrics and operational dashboards.
How deep it goes
Explains histogram aggregation, dimensional labels, and the limits of averages, but stops short of defining the scoring algorithm.
Examples
Uses Data Service search metrics, relevance-score histograms, dashboard distributions, and trace-based automated scoring.
What you can do after
Diagnose retrieval degradation by comparing score distributions across workflows, indexes, and question categories.

Read how Designing AI Systems explains relevance →

Build an AI Agent (From Scratch)

Relevance is the degree to which an answer focuses on the central aspects of a user's question without unrelated content.

How it is used
Evaluates whether an agent answers the user’s core question directly instead of producing unrelated, evasive, or incomplete content.
How deep it goes
Defines binary PASS/FAIL criteria and embeds them in a rubric-based evaluator, without using partial credit for this item.
Examples
Uses user prompts, final answers, expected answers, evidence excerpts, and PASS/FAIL evaluation outputs as assessment artifacts.
What you can do after
Evaluate agent answers with a binary rubric and track answer relevancy rates alongside other quality indicators.

Read how Build an AI Agent (From Scratch) explains relevance →

Starting Data Analytics with Generative AI and Python

Relevance is the data-quality property of being pertinent to the specific business question or problem being addressed.

How it is used
Keeps text analysis tied to the specific business question and the portion of reality being examined.
How deep it goes
Defines relevance and supplies inspection questions, but stops short of prescribing a formal relevance score or algorithm.
Examples
Uses competitor reviews, outdated reviews, order-linked comments, general impressions, and duplicated complaints as checks.
What you can do after
Evaluate whether text data belongs to the business question before analyzing it.

Read how Starting Data Analytics with Generative AI and Python explains relevance →

Machines that Think

Relevance is the degree to which a response appropriately addresses the topic or requirements of an interaction.

How it is used
Distinguishes context-sensitive stakes in learning, dialogue quality, and token-to-token relationships in attention.
How deep it goes
Explains query-key dot products, softmax weights, and value contributions, while distinguishing internal token scores from response quality.
Examples
Uses dialogue quality and the attention comparison between “rises,” “sun,” and “the.”
What you can do after
Understand how relevance differs across human learning, language quality, and attention scores.

Read how Machines that Think explains relevance →

Building Reliable AI Systems

A measure of how directly an answer addresses the question or task posed.

How it is used
Evaluates how relevant generated answers are to their questions in RAG systems.
How deep it goes
Calculates question-answer embedding cosine similarity but does not detail model selection or implementation.
Examples
Uses a question-answer similarity score of 0.85 as an answer-relevance example.
What you can do after
Evaluate answer relevance by comparing question and answer embeddings with cosine similarity.

Read how Building Reliable AI Systems explains relevance →

Also covered in

What to know first

Related concepts

Questions about relevance

What is relevance?

Relevance is the degree to which information, a result, or a response pertains to a query, task, purpose, or information need.

Which Manning books explain relevance?

Elasticsearch in Action, Second Edition, LLM Evaluation and Alignment, The Foundational Ideas, MongoDB 8.0 in Action, Third Edition, Retrieval Augmented Generation, The Foundational Ideas, AI-Powered Search, AI for Everyday IT, AI Model Evaluation, Build an Advanced RAG Application (From Scratch) and 9 more.

What should I know before learning relevance?

Manning books that teach relevance usually introduce language model, retrieval-augmented generation, hallucination and similarity search first.

What is relevance part of?

Relevance is part of Ragas. Ragas is an open-source framework and Python library for evaluating retrieval-augmented generation systems and applications.

What is relevance also called?

Relevance is also called relevancy.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.