liveBook · Manning

Term frequency

As explained in Just Use Postgres!

Term frequency is the number of occurrences of a term in a document or text representation.

Term frequency counts how many times a term occurs in a document or text representation. In PostgreSQL full-text search, ts_rank uses term frequency and term position to rank matching documents. section 6.4

Comparing occurrences in movie descriptions

The lexemes for Ghost contain 'ghost':1,2,15, showing three occurrences. A Girl Walks Home Alone at Night contains 'ghost':13, showing one. Frequency alone does not determine the final order: ts_rank also considers position, while ts_rank_cd considers term density. section 6.4

Listing 6.11 Ranking search results with the ts_rank function
SELECT id, name, vote_average, 
  ts_rank(lexemes, to_tsquery('ghosts')) AS search_rank
FROM omdb.movies
WHERE lexemes @@ to_tsquery('ghosts')
ORDER BY search_rank DESC, vote_average DESC NULLS LAST LIMIT 10;

Run a PostgreSQL query that calculates search_rank with ts_rank, then compare ranking output against the observed lexeme counts. section 6.4

How Just Use Postgres! covers it

How deep it goes
Explains occurrence counts and connects them to ts_rank, positions, density, weights, and document-length normalization.
Examples
Compares 'ghost' occurrences in two movie lexeme vectors and ranks matching movies with ts_rank.
What you can do after
Apply it via PostgreSQL by inspecting lexemes and calculating search rankings with ts_rank.
How it connects
Connects term frequency with full-text search, tsvector, tsquery, relevance ranking, term position, and term density.

Also explained in

Compare how 5 books explain term frequency →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Just Use Postgres!. The section links point to the parts of the book it draws on.