liveBook · Manning

Local-model inference

Also called local inference

Local-model inference is the execution of a trained model on hardware or infrastructure controlled by the user or deploying organization rather than through a remote service.

How 2 Manning books explain it

Vibe Engineering

Developer-local inference is the execution of an AI model on a developer's own workstation rather than on a remote serving system.

How it is used
Supplies developer-local coding assistance where privacy, offline access, completion latency, or quota pressure make remote inference less suitable.
How deep it goes
Explains hardware and model selection, runtime setup, task limits, and subscription economics, with practical memory and throughput examples.
Examples
Uses an Ollama coding-model setup, a TypeScript debounce prompt, and a twelve-person team’s subscription-cost comparison.
What you can do after
Operate local coding assistance with Ollama and a client, choosing tasks and models against hardware, privacy, and cost constraints.

Read how Vibe Engineering explains local-model inference →

Building Specialized AI Systems

Local inference is the execution of a trained model on hardware controlled by the user or deploying organization rather than on a remote service.

How it is used
Runs fine-tuned models and interchangeable adapters privately on organization-controlled hardware.
How deep it goes
Explains adapter loading, optional merging, Modelfile creation, registration, and offline execution with named tools.
Examples
Uses finance and casual adapters, financial-analysis prompts, Ollama Modelfiles, and command-line model execution.
What you can do after
Operate/configure a fine-tuned model locally with Ollama, llama.cpp, or LM Studio.

Read how Building Specialized AI Systems explains local-model inference →

Also covered in

What to know first

Related concepts

Questions about local-model inference

What is local-model inference?

Local-model inference is the execution of a trained model on hardware or infrastructure controlled by the user or deploying organization rather than through a remote service.

Which Manning books explain local-model inference?

Vibe Engineering and Building Specialized AI Systems.

What should I know before learning local-model inference?

Manning books that teach local-model inference usually introduce prompt first.

What is the difference between local-model inference and api call?

Local-model inference is the execution of a trained model on hardware or infrastructure controlled by the user or deploying organization rather than through a remote service. An API call is a programmatic request or invocation sent through an application programming interface to retrieve data or invoke an exposed operation.

What is local-model inference a kind of?

Local-model inference is a kind of inference. Inference is the process of applying a trained model to input data to produce predictions or other outputs.

What is local-model inference also called?

Local-model inference is also called local inference.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.