liveBook · Manning

Health check endpoint

As explained in Designing AI Systems

A health endpoint is an API endpoint that reports whether a service process is running or ready to receive traffic.

A health check endpoint reports whether a service process is running or ready to receive traffic. Production deployments commonly separate these responsibilities into liveness and readiness probes. section 9.3.3

Use the probes for different lifecycle decisions:

  • The liveness endpoint returns HTTP 200 while the server process is responsive. Failure indicates conditions such as deadlock, out-of-memory failure, or another unrecoverable state, prompting a container restart. section 9.3.3
  • The readiness endpoint returns HTTP 200 only when startup is complete and the container can accept traffic. During startup or graceful shutdown, it returns 503 so routing can be withheld. section 9.3.3
  • Keep both probes shallow: checking process responsiveness avoids coupling container health to gateway or service dependencies. section 9.3.3

Deployment and shutdown

A runtime server may expose /health/live and /health/ready. While it initializes its module, GenAIPlatform instance, and gRPC connections, it remains live but unready. On termination, it reports 503, allowing in-flight requests to finish before removal from load balancing. section 9.3.3

How Designing AI Systems covers it

How deep it goes
Defines liveness and readiness behavior with HTTP statuses and lifecycle rationale, but stops short of implementation code.
Examples
Uses /health/live and /health/ready during initialization, request draining, failure recovery, and container removal.
What it assumes
Assumes familiarity with HTTP services, containers, orchestrators, and request routing.
What you can do after
Operate/configure service lifecycle checks that distinguish process failure from temporary unavailability.
How it connects
Connects liveness probes, readiness probes, container orchestration, graceful shutdown, load balancing, and gRPC connections.

Also explained in

Compare how 7 books explain health check endpoint →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.