A vector database, or vector store, holds embeddings and retrieves items whose vectors are similar to a query vector. In a web application, this lets a user search for relevant documents by meaning rather than by matching only the words in a query. Vector stores can hold embeddings of whole documents or smaller text chunks. section 6.2.2 section 6.2
A document-retrieval pipeline has four practical steps:
- Ingest documents from files, APIs, databases, or user uploads. Split longer text into chunks, retaining metadata that identifies each chunk. section 6.2 section 6.2.1
- Generate an embedding for each document or chunk, then store it in a vector store. Keep source information with the stored content so retrieved results have context. section 6.2 section 7.2.4
- Embed a user's query with the same model used for the stored content. Search for similar vectors and return the most relevant documents or answers; cosine similarity and Euclidean distance are two comparison methods. section 5.4.2
- For retrieval-augmented generation (RAG), pass retrieved chunks to a prompt as context for the answer, and retain the source documents for the response. section 7.2.4
Start with an in-memory search
LangChain.js can create a MemoryVectorStore from short texts, metadata, and OpenAIEmbeddings. Calling similaritySearch("hello world", 1) returns the closest matching Document in the illustrated example. This is a low-configuration way to test ingestion and search, but the store is ephemeral. section 6.2.2
Listing 6.6 Performing similarity search
MemoryVectorStore.fromTexts(
["Hello world", "Bye bye", "hello nice world"], #1
[{ id: 2 }, { id: 1 }, { id: 3 }], #2
new OpenAIEmbeddings({apiKey}) #3
);
console.log(
await vectorStore.similaritySearch("hello world", 1)); #4
/* Example output:
[
Document {
pageContent: "Hello world",
metadata: { id: 2 }
}
]
*/
The listing shows the two essential calls: create a store from texts and embeddings, then request one similar document. section 6.2.2
Choose storage to match the application. Dedicated options include Pinecone and Milvus; databases such as PostgreSQL with pgvector can also provide vector search. LangChain.js has adapters for stores including Supabase, Pinecone, and Chroma. For a web app that must reuse its index, HNSWLib can save it to the filesystem and load it later. Indexing a large document collection is best done offline rather than during a user request. section 5.4.2 section 7.2.4
Listing 7.8 Indexing documents using a vector store
import { HNSWLib } from '@langchain/community/vectorstores/hnswlib';
async indexDocumentsFromDirectory(documentDirectory) {
const documents = [];
// Find all PDF files in the directory
const pdfFiles = fs
.readdirSync(documentDirectory)
.filter((file) =>
path.extname(file).toLowerCase() === '.pdf')
.map((file) =>
path.join(documentDirectory, file)); #1
// Process each PDF document
for (const filePath of pdfFiles) { #2
const processedDocs = await this.processDocument(filePath);
documents.push(...processedDocs);
}
// Create vector store from processed documents
const vectorStore =
await HNSWLib.fromDocuments(documents, this.embeddings); #3
console.log(`Indexed ${documents.length}
document chunks from ${pdfFiles.length} documents`);
return vectorStore;
}
A PDF indexing routine processes files into document chunks and passes them with an embedding model to HNSWLib.fromDocuments. section 7.2.4
For a multi-user knowledge-base app, decide whether users share a vector store or receive dedicated stores. Treat that choice alongside secure document handling, scalability, and maintainability—not just search speed. section 11.4
