In retrieval-augmented generation (RAG), an index organizes selected source data into searchable fields so relevant information can be retrieved for a response. A vector index can store embeddings alongside content and metadata. section 7.0 section 8.3
Plan the index before ingesting data:
- Choose the information to retrieve and exclude redundant data. section 8.3
- Account for source-system access, data format, cleaning, conversions, and engineering constraints. section 8.3
- Define fields for searchable content and context, such as titles, descriptions, dates, tags, and embeddings. section 8.3
Blog-post index
A Redis index named `posts` uses blog content and metadata, with a vector field configured for HNSW, FLOAT32 values, 1536 dimensions, and cosine distance. section 8.3
Table 8.1 Data items for blog posts we are interested in
|
Data
|
Description
|
|---|---|
|
URL
|
The URL to the individual blog post
|
|
Title
|
Title of the blog post
|
|
Description
|
A couple of sentences describing what that specific blog post is about
|
|
Publish date
|
Date when the post was published
|
|
Content
|
The actual content of the blog post
|
Review the blog-post data items selected for this example, from URL and title to publication date and content. section 8.3
