liveBook · Manning

Term frequency

Also called TF

Term frequency is the raw or normalized count of a term’s occurrences within a document or other text unit.

How 6 Manning books explain it

Natural Language Processing in Action, Second Edition

Normalized term frequency is term frequency divided by the number of terms in the document.

How it is used
Uses term frequency to quantify token usage within documents for numerical text representations.
How deep it goes
Defines raw and normalized frequency, computes examples, and discusses corpus weighting and logarithmic scaling.
Examples
Counts “justice” in a sentence and compares “bias” across two documents on algorithmic discrimination.
What you can do after
Apply term-frequency calculations using Python Counter and pandas to compare token frequencies.

Read how Natural Language Processing in Action, Second Edition explains term frequency →

Elasticsearch in Action, Second Edition

Term frequency is the count or normalized frequency of a term's occurrences within a document or field.

How it is used
Supports Elasticsearch relevance ranking by measuring how often a query term occurs in a matching document or field.
How deep it goes
Defines TF, compares TF-IDF with BM25, and shows score inspection while deferring custom similarity algorithms.
Examples
Uses Java book titles and an explained movie-title match query for “Lord.”
What you can do after
Diagnose/debug relevance scores with explain or _explain by inspecting term-frequency and related scoring details.

Read how Elasticsearch in Action, Second Edition explains term frequency →

AI-Powered Search

Term frequency is the count or within-document frequency of a term's occurrences, used to measure how strongly a document matches that term.

How it is used
Supports keyword relevance ranking by estimating how strongly a document matches a query term.
How deep it goes
Explains raw counts, cosine and dot-product effects, normalized TF, TF-IDF, and BM25 controls without deriving a complete search engine.
Examples
Uses apple-juice vectors and a three-document cat-in-the-hat collection to compare counts, normalized TF, TF-IDF, and BM25 rankings.
What you can do after
Apply it via Python to build term vectors, calculate normalized TF, and compare ranking behavior with dot product or BM25.

Read how AI-Powered Search explains term frequency →

Build an Advanced RAG Application (From Scratch)

Term frequency is the count or normalized proportion of times a term occurs within a document.

How it is used
Ranks keyword-search results by measuring a query term's presence within each document.
How deep it goes
Defines raw and normalized frequency, then implements counting inside TF-IDF without developing broader weighting theory.
Examples
Uses a fox-count equation and a six-sentence Python corpus containing terms such as learning and networks.
What you can do after
Build a basic TF-IDF keyword ranker by counting query terms in tokenized documents.

Read how Build an Advanced RAG Application (From Scratch) explains term frequency →

AI Agents in Action, Second Edition

Term frequency is the frequency of a term in a document, commonly normalized by the document's total number of words.

How it is used
Supports a simple starting point for document vectorization and vector similarity search before semantic retrieval.
How deep it goes
Defines frequency and places it within TF-IDF, but defers detailed formula derivation and semantic representation internals.
Examples
Uses a sentence about a blue sky and a collection of sentences about skies, animals, and breakfast foods.
What you can do after
Apply it via scikit-learn’s TfidfVectorizer to convert documents into sparse vectors for similarity comparison.

Read how AI Agents in Action, Second Edition explains term frequency →

Just Use Postgres!

Term frequency is the number of occurrences of a term in a document or text representation.

How it is used
Ranks PostgreSQL full-text search results for a movie recommendation service.
How deep it goes
Explains occurrence counts and connects them to ts_rank, positions, density, weights, and document-length normalization.
Examples
Compares 'ghost' occurrences in two movie lexeme vectors and ranks matching movies with ts_rank.
What you can do after
Apply it via PostgreSQL by inspecting lexemes and calculating search rankings with ts_rank.

Read how Just Use Postgres! explains term frequency →

Also covered in

What to know first

Related concepts

Questions about term frequency

What is term frequency?

Term frequency is the raw or normalized count of a term’s occurrences within a document or other text unit.

Which Manning books explain term frequency?

Natural Language Processing in Action, Second Edition, Elasticsearch in Action, Second Edition, AI-Powered Search, Build an Advanced RAG Application (From Scratch), AI Agents in Action, Second Edition and Just Use Postgres!.

What should I know before learning term frequency?

Manning books that teach term frequency usually introduce embedding, similarity search and vocabulary first.

What is term frequency a kind of?

Term frequency is a kind of frequency. Frequency is the number or rate at which an item, event, outcome, or cycle occurs within a specified collection or interval.

What is term frequency part of?

Term frequency is part of term frequency-inverse document frequency. Term frequency-inverse document frequency (TF-IDF) is a text-weighting scheme that combines a term’s frequency in a document with its inverse frequency across a document collection.

What is term frequency also called?

Term frequency is also called TF.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.