liveBook · Manning

Co-occurrence

As explained in AI-Powered Search

Co-occurrence is the occurrence of two terms within the same document or contextual unit.

Co-occurrence measures how often two terms appear in the same document, contextual unit, or user activity. In search, unusually frequent shared contexts can indicate semantic relatedness: “pain” and “ibuprofen” co-occur more meaningfully than either term and common words such as “the” or “and.” section 5.4.4 section 6.3.1

A practical co-occurrence pipeline turns search signals into related-term scores:

  • Compare raw co-occurrence with individual term counts using pointwise mutual information (PMI), which reduces the influence of generally popular terms. section 6.3.1

Related product and topic queries

A query-log aggregation finds 244,876 keyword pairs searched by at least one common user. Examples include “iphone 4s” with “iphone,” “laptop” with “hp laptop,” and “bose” with “beats.” Raw counts identify popular pairs, while PMI distinguishes association from simple popularity; sparse pairs still require caution because low shared-user counts can introduce noise. section 6.3.1

How AI-Powered Search covers it

How deep it goes
Explains document and user-based counts, PMI adjustment, composite ranking, and sparsity limits without deriving the full statistical theory.
Examples
Uses document terms, health-related products, shared-user queries, and product pairs such as “iphone 4s” and “iphone.”
What it assumes
Assumes searchable documents or query signals and basic grouping, joining, counting, and ranking operations.
What you can do after
Apply it via SQL and search analytics to generate related-term pairs and rank them with co-occurrence, PMI, or both.
How it connects
Connects co-occurrence to semantic relatedness, distributional hypothesis, query logs, pointwise mutual information, and knowledge graphs.

Also explained in

Compare how 3 books explain co-occurrence →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI-Powered Search. The section links point to the parts of the book it draws on.