liveBook · Manning

Self-hosted production

As explained in AI Governance

A self-managed GenAI deployment is an arrangement in which an organization hosts and operates the generative-AI system and its supporting processes.

A self-hosted GenAI deployment keeps model execution, data handling, and operational safeguards on organization-managed infrastructure. Teams control the end-to-end path, from a custom user interface through input and output processing to administrative policies. section 2.4.1

A typical deployment uses:

  • A pre-call pipeline to validate and format inputs, add context, anonymize sensitive data, enforce token limits, filter prompts, and apply rate controls. section 2.4.1
  • A post-call pipeline to filter and format outputs, redact sensitive information, log responses, detect hallucinations, and translate outputs into downstream actions where needed. section 2.4.1
  • An administrative control plane for prompts, policies, output limits, role-based access, and deployment settings; logging, telemetry, and inference artifacts remain local unless external systems are integrated. section 2.4.1
  • Zero Trust authentication and authorization between every subsystem, even when components share the organization’s infrastructure. section 2.4.1
Figure 2.5 Shared responsibilities across organizations (client, vendor, LLM provider) in the Self-Hosted Model adoption model: This architecture assumes the LLM (e.g., Mistral, Llama 3) is deployed on organization-managed infrastructure (e.g., cloud or on-prem). All inference traffic, safety filtering, and post-processing are managed internally. In multi-tenant systems, organizations often share centralized model serving and safety services but must enforce tenant-scoped access controls and storage partitioning for RAG, prompts/secrets, and logs to avoid cross-tenant data exposure.

The architecture illustrates organization-managed model hosting and highlights tenant-scoped access and storage partitioning as safeguards against cross-tenant exposure. section 2.4.1

How AI Governance covers it

How deep it goes
Details interface, pre-call and post-call pipelines, and administrative controls, including specific input and output safeguards.
Examples
Uses custom web portals or embedded workflow interfaces and cites Mistral and Llama 3 as model examples.
What you can do after
Operate a deployment with configurable input and output safeguards, access policies, and local logging.
How it connects
Connects deployment operations to Zero Trust, role-based access control, prompt filtering, and audit logging.

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.