Ollama is a local runtime and serving tool for downloading, running, and exposing quantized language models on a laptop. It packages model weights, configuration, and data through a Modelfile, optimizes GPU use, and serves models through a local API and chat interface. section 12.2
Operational characteristics shape the main deployment tradeoffs:
- Supports community GGUF models from the Hugging Face Hub, as well as text, vision, multimodal, and embedding models. section 12.2
- Downloads a model automatically on its first run, then starts the local command-line chat interface on subsequent runs. section 12.2
- Uses quantization to reduce memory and compute requirements; typical RAM guidance is 8 GB for 7B–8B models, 14–16 GB for 13B, and 32 GB for 33B. section 12.2
- Runs on Windows, macOS, and Linux, with a Docker image available for deployment. section 12.2
- Exposes local REST endpoints for chat and model management, allowing client applications to use the hosted model. section 12.7
Run a model and call its API
Use `ollama run deepseek-r1:8B` to download the model if necessary and begin a local chat. A client can instead POST messages to `http://localhost:11434/api/chat` with a selected model such as `mistral:latest`.
After downloading, Ollama can operate fully offline, keeping inference on the laptop. Privacy still requires local controls: the updater checks an external endpoint periodically, while plaintext prompt history and server logs remain on the installation. section 12.2.2
