liveBook · Manning

Retrieval ranking

Retrieval ranking is the ordering of candidate items according to their estimated relevance to a query.

How 9 Manning books explain it

LLM Evaluation and Alignment, The Foundational Ideas

Ranking is the ordering of alternatives according to a score, preference, relevance, or other criterion.

How it is used
Uses relevance ordering to select and prioritize documents for retrieval-augmented generation.
How deep it goes
Explains rank-aware evaluation through precision at a cutoff and mean average precision, with a worked retrieval example.
Examples
Uses a financial research query whose relevant earnings and risk documents appear third and fourth among five results.
What you can do after
Evaluate a retrieval ranking using precision@k and MAP, interpreting the results alongside context placement.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains retrieval ranking →

Retrieval Augmented Generation, The Foundational Ideas

Ranking is the ordering or filtering of retrieved candidates according to a relevance score, including ordering passages by relevance to a query.

How it is used
Prioritizes retrieved candidates before generation so relevant evidence is less likely to be missed.
How deep it goes
Explains pipeline placement, query-aware filtering and reordering, and a unified ranking-generation training format; does not derive a general relevance-scoring formula.
Examples
Uses contract-article compression and question–passage training examples to show document prioritization and relevance judgments.
What you can do after
Apply it via LLMLingua to filter and reorder context, or evaluate RankRAG as an alternative to a separate reranker.

Read how Retrieval Augmented Generation, The Foundational Ideas explains retrieval ranking →

AI-Powered Search

Search ranking is the process of ordering retrieved items according to their estimated usefulness or relevance to a query.

How it is used
Orders search results by estimated relevance after matching, placing the strongest matches first.
How deep it goes
Explains scoring, filtering, and Solr query controls, while treating domain-specific ranking factors as an introduction rather than an exhaustive guide.
Examples
Uses a children’s book query and examples from restaurant, news, and e-commerce search.
What you can do after
Configure Solr queries and boost queries to separate document filtering from relevance scoring.

Read how AI-Powered Search explains retrieval ranking →

AI Recommender Systems

The process of ordering candidate items according to a relevance or preference score.

How it is used
Orders filtered candidates by final scores to select recommendations while balancing multiple objectives.
How deep it goes
Implements weighted scoring and top-k sorting, then notes advanced objectives without detailing their algorithms.
Examples
Compares movie recommendations ranked with different similarity and popularity weights.
What you can do after
Apply weighted ranking by combining similarity and popularity scores, sorting candidates, and selecting the top k.

Read how AI Recommender Systems explains retrieval ranking →

Data Without Labels

Rank is an ordered position assigned to an observation or value relative to others.

How it is used
Represents pairwise dissimilarities by relative order when subjective assessments make absolute distances unsuitable.
How deep it goes
Defines rank-based distances in nonmetric MDS and contrasts them with metric MDS, without detailing a ranking algorithm.
Examples
Ranks six pairwise distances among points A–D, with C–D lowest and B–C highest.
What you can do after
Understand when to preserve relative distance order instead of using absolute distance values in nonmetric MDS.

Read how Data Without Labels explains retrieval ranking →

Just Use Postgres!

Relevance ranking orders text-search results according to how closely their contents match a query.

How it is used
Deploys relevance ranking in a Postgres movie recommendation search to order matching results more accurately.
How deep it goes
Explains ts_rank, ts_rank_cd, lexeme frequency, position, density, and field weighting through SQL examples, without deeper scoring theory.
Examples
Demonstrates movie searches for “ghosts,” lexeme frequency comparisons, ts_rank output, weighted fields, and ghosts:A filtering.
What you can do after
Apply it via Postgres by calculating ts_rank, ordering by search_rank, weighting fields, and filtering by weights.

Read how Just Use Postgres! explains retrieval ranking →

AI Model Evaluation

Ordering multiple outputs from best to worst according to their relative quality or utility.

How it is used
Evaluates whether retrieved or recommended items are ordered by relevance, using offline metrics or relative judgments among candidate outputs.
How deep it goes
Defines relative ordering and works through offline movie-recommendation evaluation, including Precision@K, but does not derive other ranking metrics.
Examples
Uses held-out movie ratings to assess top-K recommendations and considers ranking multiple summaries, recommendations, or agent plans.
What you can do after
Evaluate ranked movie recommendations offline by comparing top-K results with held-out movies rated 4 or higher.

Read how AI Model Evaluation explains retrieval ranking →

Outlier Detection in Python

An ordering of observations from lowest to highest or highest to lowest according to their values.

How it is used
Compare detector score orders and convert naturally ordered identifiers into processing-friendly positions.
How deep it goes
Uses ranks for score correlation, high-score filtering, and identifier conversion without defining a general ranking algorithm.
Examples
Uses detector scores on abalone data and sequential employee IDs as ranking examples.
What you can do after
Evaluate detector agreement by comparing score ranks and focusing analysis on high-scoring records.

Read how Outlier Detection in Python explains retrieval ranking →

Designing AI Systems

A rank is the ordinal position of an item in an ordered list of search results.

How it is used
Orders retrieved items and combines keyword and vector result lists into a single ranked list.
How deep it goes
Defines rank fusion with its formula and default constant, and contrasts PostgreSQL ts_rank with BM25.
Examples
Uses PostgreSQL tsvector and GIN indexing, and illustrates fusion with a result ranked highly in both lists.
What you can do after
Apply it via Reciprocal Rank Fusion to merge keyword and vector results without normalizing their scores.

Read how Designing AI Systems explains retrieval ranking →

Also covered in

What to know first

Related concepts

Questions about retrieval ranking

What is retrieval ranking?

Retrieval ranking is the ordering of candidate items according to their estimated relevance to a query.

Which Manning books explain retrieval ranking?

LLM Evaluation and Alignment, The Foundational Ideas, Retrieval Augmented Generation, The Foundational Ideas, AI-Powered Search, AI Recommender Systems, Data Without Labels, Just Use Postgres!, AI Model Evaluation, Outlier Detection in Python and 1 more.

What should I know before learning retrieval ranking?

Manning books that teach retrieval ranking usually introduce language model, precision, embedding and context window first.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.