liveBook · Manning

Ollama

As explained in Building Specialized AI Systems

Ollama is a tool for packaging, managing, and running large language models locally.

Ollama packages models into lightweight bundles that run locally, keeping inference offline for privacy, lower latency, and control over outputs. section 2.2

A local Ollama workflow is:

  • Create a Modelfile describing the base model and adapter. section 2.2
  • Build and register the model locally. section 2.2
  • Run the model interactively and offline. section 2.2

Private financial assistant

A Modelfile can reference llama2 and a finance adapter; `ollama create finance-model -f Modelfile` registers it, and `ollama run finance-model` starts local use.

Figure 2.2 LoRA adapter swapping for different domains or use cases. A single frozen foundation model serves as the shared backbone for multiple lightweight domain adapters. Each adapter adds a small set of trainable weights alongside the frozen layers — less than 0.1% of the base model's parameters — and can be swapped in or out without modifying the underlying model.

Shows how one frozen foundation model can support interchangeable lightweight adapters for different domains or tasks. section 2.2

How Building Specialized AI Systems covers it

How deep it goes
Explains packaging, Modelfile creation, registration, and execution, but defers detailed Ollama command syntax beyond the basic workflow.
Examples
Uses a finance adapter, Modelfile, and `ollama create` and `ollama run` commands to demonstrate a local financial assistant.
What it assumes
Assumes a base model and, when adapting behavior, saved adapter weights available for local loading.
What you can do after
Operate/configure a packaged model locally by declaring an adapter, registering the model, and running it offline.
How it connects
Connects local inference with adapters, parameter-efficient fine-tuning, llama.cpp, and LM Studio.

Also explained in

Compare how 10 books explain Ollama →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Building Specialized AI Systems. The section links point to the parts of the book it draws on.