A leaked model-provider key can expose powerful models, internal data, and substantial spending limits. Many exposed secrets remain valid for years, extending the risk beyond the initial leak. section 4.3.5
Store API keys and tokens in a centralized secrets manager, such as AWS Secrets Manager or HashiCorp Vault. Treat the vault as a control that protects the full system flow: authenticate and authorize each component, encrypt service communications, and segment networks so only necessary components can connect. section 2.4.1
Leaked key, expanded access
An employee’s committed xAI API key reportedly provided access to at least 60 private and unreleased language models, illustrating how one exposed credential can reach beyond a single application. section 4.3.5
Figure 2.5 Shared responsibilities across organizations (client, vendor, LLM provider) in the Self-Hosted Model adoption model: This architecture assumes the LLM (e.g., Mistral, Llama 3) is deployed on organization-managed infrastructure (e.g., cloud or on-prem). All inference traffic, safety filtering, and post-processing are managed internally. In multi-tenant systems, organizations often share centralized model serving and safety services but must enforce tenant-scoped access controls and storage partitioning for RAG, prompts/secrets, and logs to avoid cross-tenant data exposure.

See how secret storage fits into a self-hosted deployment, where tenant-scoped access controls and partitioned storage help prevent cross-tenant exposure. section 2.4.1







