A self-hosted GenAI deployment keeps model execution, data handling, and operational safeguards on organization-managed infrastructure. Teams control the end-to-end path, from a custom user interface through input and output processing to administrative policies. section 2.4.1
A typical deployment uses:
- A pre-call pipeline to validate and format inputs, add context, anonymize sensitive data, enforce token limits, filter prompts, and apply rate controls. section 2.4.1
- A post-call pipeline to filter and format outputs, redact sensitive information, log responses, detect hallucinations, and translate outputs into downstream actions where needed. section 2.4.1
- An administrative control plane for prompts, policies, output limits, role-based access, and deployment settings; logging, telemetry, and inference artifacts remain local unless external systems are integrated. section 2.4.1
- Zero Trust authentication and authorization between every subsystem, even when components share the organization’s infrastructure. section 2.4.1
Figure 2.5 Shared responsibilities across organizations (client, vendor, LLM provider) in the Self-Hosted Model adoption model: This architecture assumes the LLM (e.g., Mistral, Llama 3) is deployed on organization-managed infrastructure (e.g., cloud or on-prem). All inference traffic, safety filtering, and post-processing are managed internally. In multi-tenant systems, organizations often share centralized model serving and safety services but must enforce tenant-scoped access controls and storage partitioning for RAG, prompts/secrets, and logs to avoid cross-tenant data exposure.

The architecture illustrates organization-managed model hosting and highlights tenant-scoped access and storage partitioning as safeguards against cross-tenant exposure. section 2.4.1
