BM25 is a probabilistic ranking function for keyword search. It scores documents using query-term frequency, term importance across the corpus, document length, and the hyperparameters k1 and b. section 2.1.1
BM25 is useful when exact lexical matching is the intended signal, but it has clear boundaries:
- Weights rarer query terms more heavily and controls the effect of repeated terms through term-frequency saturation. section 2.1.1 section 2.1.3
- Normalizes for document length, helping compare documents without treating long documents as automatically more relevant. section 2.1.1
- Matches terms discretely: automobile, car, and vehicle remain unrelated unless preprocessing adds synonym expansion, lemmatization, or typo correction. section 2.1.1 section 2.1.3
Keyword mismatch in a toy corpus
For the query “automobile maintenance,” BM25 ranks “The automobile industry is booming” first with 1.17 and gives “I bought a new car today” a score of 0.00 because it shares no query terms. Documents containing maintenance or automobile receive nonzero scores, while semantic similarity is not inferred. section 2.1.3
Inspect the BM25 scoring equation, including inverse document frequency, term frequency, document length, average length, k1, and b. section 2.1.1






