TF-IDF ranks documents by combining two signals: how often a query term appears in one document and how rare that term is across the collection. Frequent terms in a specific document receive more weight, while terms common across many documents receive less. section 3.4.2
Ranking Java book titles
For a search for Java, “Mastering Java: Learning Core Java and Enterprise Java With Examples” contains Java three times, while “Effective Java” and “Head First Java” contain it once. TF-IDF therefore favors higher term frequency, while inverse document frequency reduces the value of terms appearing broadly across the collection.
Document length also affects practical relevance. A term repeated in a short title can be more useful than the same term appearing in a much longer field, so field choice matters when comparing matches. section 3.4.1
Use TF-IDF as a baseline for understanding Elasticsearch relevance:
- Elasticsearch applies similarity algorithms per field through mappings and can select or customize similarity when the default is unsuitable. section 3.4.2
- BM25 improves on TF-IDF with nonlinear term-frequency saturation and document-length normalization; BM25 is Elasticsearch’s default. section 3.4.2
