liveBook · Manning

FastAPI

As explained in LLMs in Production

FastAPI is a Python web framework for building HTTP APIs with type-based request validation and automatic interface documentation.

FastAPI is a practical choice when an automated model-serving framework is too restrictive. It lets a Python service expose a model through an HTTP API while leaving deployment behavior customizable. section 3.4.8

Serving a coding model locally

A small FastAPI service exposes a DeciCoder model from Python. Start it with `python server.py`, then send `{"prompt":"def hello_world(name):"}` to `http://localhost:8000/generate` with curl; the response can be a completed Python function. section 10.1

A production-facing FastAPI deployment service should:

  • Automate boilerplate while preserving customization where framework-provided methods are insufficient. section 3.4.8
  • Bridge model serving with CI/CD pipelines, tests, health checks, rollbacks, containers, resource configuration, and secret management. section 3.4.8

How LLMs in Production covers it

How deep it goes
Demonstrates a local model-serving endpoint and discusses production integration, but does not develop FastAPI internals.
Examples
Uses a DeciCoder endpoint, Python server, stop tokens, and a curl request generating a Hello World function.
What it assumes
Assumes Python familiarity and model-serving context while demonstrating the API workflow from a simple endpoint.
What you can do after
Operate a local model API by running a Python server and calling its generation endpoint with curl.
How it connects
Connects model serving with MLOps, DevOps, CI/CD, Kubernetes, HTTP APIs, and retrieval-augmented generation.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

Compare how 5 books explain FastAPI →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.