Indexing turns raw documents into a searchable representation, preparing a corpus for sparse retrieval such as TF-IDF or BM25. It is analogous to cataloging books so readers can find them by title, topic, author, or subject. section 6.2.1
Indexing
Indexing is the process of transforming raw documents into a searchable representation.
How LLM Evaluation and Alignment, The Foundational Ideas covers it
- How deep it goes
- Defines indexing as creating a searchable representation and compares it to library cataloging.
- Examples
- Uses library cataloging as an analogy for finding documents by title, topic, author, or subject.
- What you can do after
- Understand indexing as the corpus-preparation step required before sparse retrieval can work.
- How it connects
- Connects indexing to sparse retrieval, TF-IDF, BM25, and corpus processing.
Also explained in
-
Grokking Data StructuresAn array index is the position used to identify an element within an array.
-
Modern C, Third EditionArray indexing is the C operation of selecting an array element with an index expression, such as A[i].
-
Learn Rust in a Month of LunchesIndexing selects elements in an ordered collection by their numeric positions.
-
Pro Angular 16An array index is a numeric position used to access an element in an array.
-
Generative AI in ActionData indexing is the organization of stored data into searchable structures that support efficient information retrieval.
-
Modern CAn index is an integer position used to identify an item in an array.
-
Build AI-Enhanced Web AppsDocument indexing is the organization of document content into searchable structures for efficient information retrieval.
-
Deep Learning with R, Third EditionIndexing is the mapping of text tokens to integer positions in a vocabulary.
-
Deep Learning with PyTorch, Second EditionSelecting tensor elements or slices by using tensor values as indices or masks.
-
Natural Language Processing in Action, Second EditionIndexing is the construction of auxiliary structures that organize document content for efficient retrieval.
-
Vibe EngineeringProject indexing is the process of analyzing and cataloging project files to support efficient code search and contextual retrieval.
-
Deep Learning with Python, Third EditionThe stage of text preprocessing that maps tokens to integer indices for numeric model input.
-
Julia as a Second LanguageAn index is an integer position used to identify an element within an ordered collection.
-
AI-Powered SearchIndexing is the process of ingesting and organizing content into a searchable index.
-
Learn Generative AI with PyTorchThe use of bracket notation to access selected elements or subregions of a tensor.
-
Grokking SimplicityA numeric position identifying an element within an indexed array.
-
Python Workout, Second EditionAn index is a numeric position identifying an element within an ordered sequence.
-
The Well-Grounded Rubyist, Fourth EditionAn index is a numeric or key-based identifier for an element in a collection, including an ordinal position in an ordered collection.
-
Grokking Software ArchitectureThe creation and maintenance of auxiliary structures that help a database locate records efficiently.
-
Context EngineeringIndexing organizes information into a structure that supports efficient search and retrieval.
Compare how 22 books explain indexing →
Related concepts
- arrayused by · An array is an ordered collection of elements organized in one or more dimensions and accessible by position or index.
- partial indexincludes · A partial index is a database index containing entries only for records that satisfy a specified predicate.
- covering indexincludes · A covering index is an index containing all columns needed to evaluate and return a query, avoiding access to the base table.
- homogeneous string mapkind of · A homogeneous string map is a map with string keys whose values all have the same type or kind.
This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.





















