liveBook · Manning

Data service

As explained in Designing AI Systems

A data service is a reusable platform service that ingests documents, generates vector embeddings, and supports semantic search over organizational knowledge.

A data service turns organizational documents into searchable knowledge for AI applications. Instead of rebuilding parsing, chunking, embedding, and storage for every assistant, teams use a shared service to retrieve relevant context and ground responses in current policies, products, and procedures. section 5.0 section 1.4

The service organizes knowledge into isolated indexes. Each index has its own embedding model, chunking strategy, and metadata schema, allowing teams to configure retrieval independently. Chunking matters: splitting critical information across chunks can prevent retrieval of a complete answer. section 5.7

A reusable retrieval pipeline separates document processing from application logic:

  • Ingest documents through format-specific parsers, extract text and structure, and divide the content into chunks. On reingestion, replace old chunks with those from the updated document. section 5.7
  • Generate embeddings through the Model Service, reusing its provider abstraction, cost tracking, fallback, and retry handling. Store chunks, vectors, and metadata behind a vector-store interface. section 5.7
  • Search for relevant chunks and return ranked results with scores, document IDs, and metadata. Hybrid search can combine vector and keyword rankings using reciprocal rank fusion, avoiding score normalization. section 5.7

The gRPC contract exposes index management, ingestion jobs, document management, search, and hybrid search. A thin Python SDK client translates calls and responses into domain objects. Both search methods return SearchResult lists, so workflows can switch retrieval methods without changing result handling. section 5.6

Listing 5.24 Data service gRPC contract
service DataService {
    // Index management
    rpc CreateIndex(CreateIndexRequest) returns (IndexResponse);
    rpc GetIndex(GetIndexRequest) returns (IndexResponse);
    rpc ListIndexes(ListIndexesRequest) returns (ListIndexesResponse);
    rpc DeleteIndex(DeleteIndexRequest) returns (DeleteIndexResponse);
 
    // Document ingestion
    rpc IngestDocument(IngestDocumentRequest) returns (IngestJobResponse);
    rpc GetIngestJob(GetIngestJobRequest) returns (IngestJobResponse);
 
    // Document management
    rpc ListDocuments(ListDocumentsRequest) returns (ListDocumentsResponse);
    rpc GetDocument(GetDocumentRequest) returns (DocumentMetadataResponse);
    rpc DeleteDocument(DeleteDocumentRequest) returns
        (DeleteDocumentResponse);
 
    // Search
    rpc Search(SearchRequest) returns (SearchResponse);
    rpc HybridSearch(HybridSearchRequest)
        returns (SearchResponse);    #A
 
    // Extension registration
    rpc RegisterParser(RegisterParserRequest) returns (RegisterParserResponse);
    rpc RegisterChunkingStrategy(RegisterChunkingStrategyRequest)
        returns (RegisterChunkingStrategyResponse);    
}

Inspect the service contract for the operations exposed across the service boundary. section 5.6

Grounding a patient intake assistant

A workflow searches a patient-procedures index with the user’s question, joins the returned text into model context, and returns document IDs as sources. The application chooses the index, result count, score threshold, and context format; the service handles retrieval infrastructure. section 2.1.3 section 5.6

Listing 2.3 Integrating organizational knowledge through Data Service
relevant_info = platform.data.search(    #A
    query=question,    #A
    index_name="patient_procedures"    #A
)
relevant_text = "\n".join(r.text for r in relevant_info)    #A
 
response = platform.models.chat(
    model="gpt-4o",
    system_prompt_name="patient_intake_assistant",
    messages=[    #B
        {"role": "system",    #B
         "content": f"Relevant information:\n{relevant_text}"},    #B
        {"role": "user", "content": question}],    #B
)
return {
    "response": response.content,
    "sources": [r.document_id for r in relevant_info]    #C
}

See the Python search, model call, and source-document return in one workflow. section 2.1.3

A data service complements session management rather than replacing it: retrieved documents provide organizational knowledge, while conversation history supplies interaction context. For storage, the illustrated pgvector backend keeps vectors alongside chunk text and metadata in PostgreSQL and uses IVFFlat for approximate similarity search. section 1.4 section 5.7

How Designing AI Systems covers it

How deep it goes
Builds ingestion, vector storage, hybrid retrieval, and service interfaces, including a pgvector implementation.
Examples
Uses a patient intake assistant to search procedure documents and return source IDs with a generated response.
What you can do after
Builds a data service from ingestion through retrieval, and applies its Python SDK in application workflows.
How it connects
Connects semantic search with embeddings, chunking, retrieval-augmented generation, session management, and model abstraction.

Also explained in

Compare how 2 books explain data service →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.