liveBook · Manning

Forward index

As explained in AI-Powered Search

A forward index is a data structure that maps each document and field to the terms contained in that document.

A forward index maps each document and field to the terms contained in that document. It is the document-oriented counterpart to an inverted index, which maps terms to documents. section 5.4.3

In a search engine, the structure supports several operations:

  • Faceting or aggregations, which show the top field values across a result set. section 5.4.3
  • Real-time traversal and ranking of semantic relationships in a document collection. section 5.4.3
  • Index-time generation through doc values in Lucene-based systems; Solr can also uninvert the inverted index in memory at query time. section 5.4.3

Document fields and terms

Suppose three documents contain job_title, desc, and skills fields. The forward index records the terms belonging to each document within each field, while the inverted index supports lookup from a field term to its matching documents. section 5.4.3

Figure 5.4 Inverted index and forward index. Documents get added to an inverted index, which maps documents to lists of terms, and to a forward index, which maps terms back to lists of documents. Having the ability to map both directions will prove important for graph traversal and relationship discovery.
figure

Compare the two index directions visually before relating them to graph traversal and relationship discovery. section 5.4.3

How AI-Powered Search covers it

How deep it goes
Defines document-to-term mapping and describes doc values and query-time uninversion, without detailing storage internals.
Examples
Uses three documents with job_title, desc, and skills fields to contrast forward and inverted mappings.
What it assumes
Assumes familiarity with indexed documents and inverted indexes while explaining forward indexes from their basic mapping.
What you can do after
Understand how search engines generate facets and support relationship traversal from document-field term data.
How it connects
Connects inverted indexes, doc values, faceting, aggregations, knowledge graphs, and graph traversal.

Also explained in

Compare how 2 books explain forward index →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI-Powered Search. The section links point to the parts of the book it draws on.