TF-IDF weights a term by combining its frequency within one document with its frequency across a document collection. It supports keyword search by scoring documents that overlap with query terms, while an inverted index makes matching documents quick to find. section 3.2
A small keyword-search implementation follows this pipeline:
- Tokenize each document into words using NLTK's word_tokenize function. section 3.2.1
- Build an inverted index that maps each word to the document indices containing it. section 3.2
Ranking a machine-learning query
For the query "machine learning," the implementation scores "Machine learning is based on algorithms" highest at 1.791759469228055. "Deep learning uses neural networks" follows at 0.6931471805599453, while documents without matching query terms receive zero.











