liveBook · Manning

Tsvector

As explained in PostgreSQL Mistakes and How to Avoid Them

tsvector is a PostgreSQL data type representing normalized searchable text as a collection of lexemes and their positions.

tsvector supports PostgreSQL full-text search by converting text into a searchable representation. Pairing it with a GIN index avoids treating long document fields like ordinary B-tree values. section 4.4

Avoid these common indexing mistakes:

  • Do not use a B-tree for arbitrary substring searches or large abstracts; long values can exceed PostgreSQL's index-row limit. section 4.4
  • Create a GIN expression index with a chosen text configuration, then query the matching tsvector expression with @@. section 4.4
  • Choose the index for the predicate: GIN suits full-text search, while a focused B-tree is smaller and faster for exact identifiers. section 4.4

Indexed title search

CREATE INDEX ON test.arxiv USING gin (to_tsvector('english', data->>'title')); Query the same expression with plainto_tsquery('english', 'modeling of hydrogen'). The resulting plan uses a Bitmap Index Scan and completes in about 5 ms for the demonstrated dataset. section 4.4

DROP INDEX test.arxiv_lower_idx;
DROP INDEX

CREATE INDEX ON test.arxiv
    USING gin (to_tsvector('english', data->>'title'));
CREATE INDEX

Inspect the executable GIN index definition for a JSON title expression. section 4.4

How PostgreSQL Mistakes and How to Avoid Them covers it

How deep it goes
Contrasts B-tree failures with a working GIN expression index and query plan, but does not detail tokenization internals.
Examples
Uses an arXiv JSON dataset, English title indexing, plainto_tsquery, EXPLAIN ANALYZE, and measured index sizes.
What you can do after
Apply it via PostgreSQL GIN indexes and diagnose full-text performance with EXPLAIN ANALYZE.
How it connects
Connects tsvector with GIN indexes, full-text search, text configurations, tsquery, B-tree indexes, and JSONB.

Also explained in

Compare how 2 books explain tsvector →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of PostgreSQL Mistakes and How to Avoid Them. The section links point to the parts of the book it draws on.