liveBook · Manning

API gateway

As explained in AI Governance

A server component that mediates requests between clients and backend services, applying routing, authentication, and other controls.

For GenAI systems, an API gateway provides a central route for vendor-model traffic, allowing organizations to sanitize data before it leaves their systems and apply access and usage controls. section 4.3.6 section 5.2.4

Operational controls can include:

  • Redact identifiers or secrets, block sensitive fields, and limit which documents enter retrieval-augmented generation. section 5.2.4
  • Require service identity, apply per-service rate limits and guardrails, and collect detailed logs. section 4.3.6
  • Monitor for unauthorized vendors or traffic that bypasses the gateway; scan repositories for unapproved endpoints and block merges that add them. section 4.3.6

A quiet caller bypasses controls

An engineer frustrated by latency deploys a direct API call from another account. Although it uses the same model, the call avoids the approved gateway and its contractual and technical controls. section 4.3.6

How AI Governance covers it

How deep it goes
Specifies redaction, guardrails, service identity, rate limits, and bypass monitoring, but not gateway internals.
Examples
Contrasts a sanctioned gateway route with an engineer's direct API call and describes sanitizing data before vendor submission.
What you can do after
Configure a central gateway to sanitize vendor-bound data, enforce service-level controls, and detect unauthorized or bypassing traffic.
How it connects
Connects to shadow AI, data loss prevention, single sign-on, retrieval-augmented generation, and data minimization.

Also explained in

Compare how 10 books explain API gateway →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.