Replace-on-reingest keeps indexed content current by assigning a stable document identifier, deleting that document’s existing chunks and vectors, then extracting, chunking, embedding, and storing the new version. section 5.3.7 section 5.7
A production implementation should preserve these properties:
- Use a caller-supplied stable document_id when available; otherwise derive an identifier from the filename. section 5.3.7
- Treat deletion, extraction, chunking, embedding, and insertion as one atomic operation so failures restore the previous indexed version. section 5.3.7
- Prefer full replacement when simplicity and correctness outweigh re-embedding unchanged sections; diff-based updates can fail when section boundaries shift. section 5.3.7
Updated support article
A revised support article is ingested with its existing document_id. The service removes its old chunks and vectors, processes the revised file, and inserts the replacement content without manual index cleanup. section 5.3.7 section 5.7
Listing 5.13 Document ingestion
def ingest_document(
self,
index_name: str,
filename: str,
file_bytes: bytes,
metadata: Optional[Dict[str, str]] = None,
document_id: Optional[str] = None #A
) -> IngestedDocument:
index = self.get_index(index_name)
document_id = document_id or generate_document_id(filename)
self.vector_store.delete_by_document(
index_name, document_id
) #B
extracted = self.pipeline.extract(filename, file_bytes)
strategy = self.chunking_strategies[
index.config.chunking_strategy
](
chunk_size=index.config.chunk_size,
chunk_overlap=index.config.chunk_overlap
) #C
chunks = strategy.chunk(extracted)
embeddings = self.embedding_generator.embed_chunks(
chunks,
model=index.config.embedding_model
) #D
self.vector_store.insert(
index_name, document_id, chunks, embeddings, metadata or {}
) #E
return IngestedDocument(
document_id=document_id,
chunk_count=len(chunks),
index_name=index_name
)Shows the ingestion service boundary and the ordered replacement pipeline, including document deletion, configurable chunking, embedding generation, and vector insertion. section 5.3.7
