liveBook · Manning

Model hoster posture

As explained in AI Governance

A model hoster is an organization that runs a generative AI model on infrastructure it controls or manages.

A model hoster runs a generative AI model on infrastructure it controls or manages, such as on-premises infrastructure, a private cloud or VPC, or a managed hosting service. The posture gives the organization greater control over model behavior, data handling, and performance, while making its team responsible for security, uptime, and compliance. section 2.1

Unlike an API integrator, which sends data to an external model and depends on its vendor for core functionality, a hoster operates the model itself. An organization can use several postures at once for different workflows. section 2.1 section 2.0

Figure 2.1 Different types of GenAI adoption. SaaS Consumers have the least control and highest vendor dependency, Model Hosters the highest control and lowest vendor dependency

Compare model hosting with other adoption postures by organizational control and vendor dependency. section 2.1

Hosting for a regulated workflow

A hospital group might deploy an open-source medical language model inside an air-gapped VPC so patient records remain within its HIPAA-controlled boundary. The added control also means the hospital must manage the model and its safeguards. section 2.1

Operating the posture requires controls across the model lifecycle and serving stack:

  • Evaluate open models before production, especially for high-risk uses, when training-data provenance or alignment is unclear. Implement and validate guardrails rather than assuming they are included. section 2.4.3
  • Clean and validate fine-tuning data, then retest safety behavior. Fine-tuning can introduce bias, poisoning, privacy violations, or degraded alignment. section 2.4.3
  • Maintain a software bill of materials, use trusted dependency registries, and patch vulnerabilities in libraries and containers. section 2.4.3
  • Partition retrieval indexes, prompts, secrets, and logs by tenant, and enforce tenant-scoped access at retrieval time. Test and monitor retrieval behavior. section 2.4.3 section 5.4.3
  • Require human review when fabricated statements about individuals cannot be tolerated; output filters and factuality checks are not complete safeguards. section 5.4.3

Privacy governance must also cover derivative artifacts such as checkpoints, embeddings, evaluation datasets, and backups. Hosters need technical evidence that privacy rights are addressed across the lifecycle, while acknowledging limits on reliable model-level unlearning and correction. section 5.5.4

How AI Governance covers it

How deep it goes
Provides governance controls for evaluation, fine-tuning, serving, and tenant isolation, while acknowledging limits on reliable model-level unlearning.
Examples
Illustrates hosting with a hospital medical model, an insurer’s fine-tuned claims assistant, and a low-latency fraud gateway.
What you can do after
Operate and configure a hosted model with evaluation, dependency management, tenant isolation, and human review safeguards.
How it connects
Connects model hosting to fine-tuning, retrieval-augmented generation, tenant isolation, privacy rights, and supply-chain security.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.