Term frequency
Term frequency is the raw or normalized count of a term’s occurrences within a document or other text unit.
Natural Language Processing in Action, Second Edition
Normalized term frequency is term frequency divided by the number of terms in the document.
- How it is used
- Uses term frequency to quantify token usage within documents for numerical text representations.
- How deep it goes
- Defines raw and normalized frequency, computes examples, and discusses corpus weighting and logarithmic scaling.
- Examples
- Counts “justice” in a sentence and compares “bias” across two documents on algorithmic discrimination.
- What you can do after
- Apply term-frequency calculations using Python Counter and pandas to compare token frequencies.
Read how Natural Language Processing in Action, Second Edition explains term frequency →
Elasticsearch in Action, Second Edition
Term frequency is the count or normalized frequency of a term's occurrences within a document or field.
- How it is used
- Supports Elasticsearch relevance ranking by measuring how often a query term occurs in a matching document or field.
- How deep it goes
- Defines TF, compares TF-IDF with BM25, and shows score inspection while deferring custom similarity algorithms.
- Examples
- Uses Java book titles and an explained movie-title match query for “Lord.”
- What you can do after
- Diagnose/debug relevance scores with explain or _explain by inspecting term-frequency and related scoring details.
Read how Elasticsearch in Action, Second Edition explains term frequency →
AI-Powered Search
Term frequency is the count or within-document frequency of a term's occurrences, used to measure how strongly a document matches that term.
- How it is used
- Supports keyword relevance ranking by estimating how strongly a document matches a query term.
- How deep it goes
- Explains raw counts, cosine and dot-product effects, normalized TF, TF-IDF, and BM25 controls without deriving a complete search engine.
- Examples
- Uses apple-juice vectors and a three-document cat-in-the-hat collection to compare counts, normalized TF, TF-IDF, and BM25 rankings.
- What you can do after
- Apply it via Python to build term vectors, calculate normalized TF, and compare ranking behavior with dot product or BM25.
Build an Advanced RAG Application (From Scratch)
Term frequency is the count or normalized proportion of times a term occurs within a document.
- How it is used
- Ranks keyword-search results by measuring a query term's presence within each document.
- How deep it goes
- Defines raw and normalized frequency, then implements counting inside TF-IDF without developing broader weighting theory.
- Examples
- Uses a fox-count equation and a six-sentence Python corpus containing terms such as learning and networks.
- What you can do after
- Build a basic TF-IDF keyword ranker by counting query terms in tokenized documents.
Read how Build an Advanced RAG Application (From Scratch) explains term frequency →
AI Agents in Action, Second Edition
Term frequency is the frequency of a term in a document, commonly normalized by the document's total number of words.
- How it is used
- Supports a simple starting point for document vectorization and vector similarity search before semantic retrieval.
- How deep it goes
- Defines frequency and places it within TF-IDF, but defers detailed formula derivation and semantic representation internals.
- Examples
- Uses a sentence about a blue sky and a collection of sentences about skies, animals, and breakfast foods.
- What you can do after
- Apply it via scikit-learn’s TfidfVectorizer to convert documents into sparse vectors for similarity comparison.
Read how AI Agents in Action, Second Edition explains term frequency →
Just Use Postgres!
Term frequency is the number of occurrences of a term in a document or text representation.
- How it is used
- Ranks PostgreSQL full-text search results for a movie recommendation service.
- How deep it goes
- Explains occurrence counts and connects them to ts_rank, positions, density, weights, and document-length normalization.
- Examples
- Compares 'ghost' occurrences in two movie lexeme vectors and ranks matching movies with ts_rank.
- What you can do after
- Apply it via PostgreSQL by inspecting lexemes and calculating search rankings with ts_rank.
Also covered in
What to know first
- embeddingAn embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
- similarity searchSimilarity search is a retrieval technique that finds or ranks items by their similarity to a query, often by comparing feature or vector representations under a similarity or distance measure.
- vocabularyA vocabulary is the finite set of distinct tokens a language-processing system can represent, often associated with identifiers.
Related concepts
- term frequency-inverse document frequencypart of · Term frequency-inverse document frequency (TF-IDF) is a text-weighting scheme that combines a term’s frequency in a document with its inverse frequency across a document collection.
- relevanceuses · Relevance is the degree to which information, a result, or a response pertains to a query, task, purpose, or information need.
- frequencykind of · Frequency is the number or rate at which an item, event, outcome, or cycle occurs within a specified collection or interval.
- document length normalizationconfigured by · Document length normalization adjusts retrieval relevance scores to account for differences in document length or term frequency.
- token countmeasures · A token count is a measure of the number of tokens present in text or processed or generated during a model or corpus operation.
Questions about term frequency
What is term frequency?
Term frequency is the raw or normalized count of a term’s occurrences within a document or other text unit.
Which Manning books explain term frequency?
Natural Language Processing in Action, Second Edition, Elasticsearch in Action, Second Edition, AI-Powered Search, Build an Advanced RAG Application (From Scratch), AI Agents in Action, Second Edition and Just Use Postgres!.
What should I know before learning term frequency?
Manning books that teach term frequency usually introduce embedding, similarity search and vocabulary first.
What is term frequency a kind of?
Term frequency is a kind of frequency. Frequency is the number or rate at which an item, event, outcome, or cycle occurs within a specified collection or interval.
What is term frequency part of?
Term frequency is part of term frequency-inverse document frequency. Term frequency-inverse document frequency (TF-IDF) is a text-weighting scheme that combines a term’s frequency in a document with its inverse frequency across a document collection.
What is term frequency also called?
Term frequency is also called TF.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.





