Ollama
Ollama is software for downloading, managing, running, and serving language models locally.
Build Applications with Local AI Models on a Mac
Ollama is a tool for downloading, running, and serving language models locally.
- How it is used
- Uses Ollama as the local language-model runtime for a private, adaptable assistant.
- How deep it goes
- Explains request flow and provides installation, startup, verification, and connection-error troubleshooting steps.
- Examples
- Uses a music-player analogy and a connection-refused error to illustrate the runtime and server.
- What you can do after
- Operate/configure Ollama to install and run local models, verify its service, and troubleshoot a refused connection.
Read how Build Applications with Local AI Models on a Mac explains Ollama →
Vibe Engineering
Ollama is a software tool for downloading, running, and serving language models locally.
- How it is used
- Runs local language-model inference for private, offline, low-latency completions and simple development tasks.
- How deep it goes
- Defines the runtime boundary and gives installation commands, but defers detailed infrastructure and model internals.
- Examples
- Uses Ollama installation commands, a Qwen coding model, a TypeScript prompt, and a local-versus-cloud configuration.
- What you can do after
- Operate/configure it by installing Ollama, pulling a coding model, verifying inference, and routing tasks across local and cloud models.
Domain-Specific Small Language Models
A local model runtime and serving tool that downloads, runs, and exposes quantized language models through an API and chat interface.
- How it is used
- Runs personal language-model assistants locally and exposes models to client applications through an API.
- How deep it goes
- Explains installation, model selection, CLI chat, API access, hardware requirements, and privacy caveats without detailing runtime internals.
- Examples
- Uses a DeepSeek-R1 8B CLI chat, a Hugging Face Hub model URL, and a local chat API request.
- What you can do after
- Operate/configure it to download and chat with a local model, or serve that model to an application through its REST API.
Read how Domain-Specific Small Language Models explains Ollama →
Server-Side WebAssembly
Ollama is a local machine-learning runtime and command-line tool for downloading and running large language models.
- How it is used
- Provides local or cluster-hosted LLM inference for SmartCMS story generation through a custom Wasm provider.
- How deep it goes
- Explains installation, model pulling, WIT contract integration, component calls, storage, and Kubernetes deployment without implementing the provider runtime.
- Examples
- Uses a TinyStories continuation model, a /api/generate endpoint, generated-story storage, and an Ollama Kubernetes Deployment and Service.
- What you can do after
- Apply it via Ollama, a custom provider, and Kubernetes manifests to generate and persist SmartCMS stories.
Build a Large Language Model (From Scratch)
A local application and serving wrapper for running large language models for inference through a command line or API.
- How it is used
- Supports local LLM inference and automated evaluation of instruction-fine-tuned model responses.
- How deep it goes
- Explains startup, model execution, API requests, and scoring code, while stopping short of training or fine-tuning internals.
- Examples
- Uses ollama run llama3, a llama prompt, Python API queries, and automated scores over 110 test entries.
- What you can do after
- Operate/configure a local Llama 3 service and query it from Python for text generation or response scoring.
Read how Build a Large Language Model (From Scratch) explains Ollama →
Building Specialized AI Systems
Ollama is a tool for packaging, managing, and running large language models locally.
- How it is used
- Runs fine-tuned models and adapters locally when privacy, data security, latency, or output control makes remote inference unsuitable.
- How deep it goes
- Explains packaging, Modelfile creation, registration, and execution, but defers detailed Ollama command syntax beyond the basic workflow.
- Examples
- Uses a finance adapter, Modelfile, and `ollama create` and `ollama run` commands to demonstrate a local financial assistant.
- What you can do after
- Operate/configure a packaged model locally by declaring an adapter, registering the model, and running it offline.
Spring AI in Action
Ollama is a tool for running and serving open-source language models locally on user hardware.
- How it is used
- Runs open-source language models locally for development and Spring AI applications.
- How deep it goes
- Explains installation, model selection, dependency setup, configuration, and hardware trade-offs without detailing model internals.
- Examples
- Uses Gemma 2B, Gemma 7B, Mistral 7B, Ollama commands, and the /api/tags endpoint.
- What you can do after
- Operate/configure Ollama locally and connect a selected model to Spring AI without cloud credentials.
AI-Powered Developer: Build software with ChatGPT and Copilot
Software for running and managing large language models locally, including through Docker-based deployments.
- How it is used
- Deploys Llama 2 locally for offline experimentation and comparison with managed LLM output.
- How deep it goes
- Explains Docker startup, model downloading, prompt refinement, and output checking, but does not describe Ollama internals.
- Examples
- Uses Docker commands, model self-description, pure-Python standard-deviation code, formula explanations, and degrees-of-freedom questions.
- What you can do after
- Operate Ollama in Docker, run Llama 2 locally, and evaluate or refine its answers against a known-correct baseline.
Read how AI-Powered Developer: Build software with ChatGPT and Copilot explains Ollama →
Data-Driven Cybersecurity
Ollama is a tool for running and interacting with large language models locally through command-line and API interfaces.
- How it is used
- Deploys local language models for privacy-conscious cybersecurity analysis and integration into custom applications or automated workflows.
- How deep it goes
- Explains local interaction, privacy, performance, customization, API integration, and command-line use, but provides no Ollama-specific code or installation procedure.
- Examples
- Demonstrates cybersecurity insight generation, local handling of sensitive data, and comparison with GUI-based LM Studio workflows.
- What you can do after
- Operate/configure it for local model use, API-connected applications, customized tasks, and privacy-sensitive cybersecurity workflows.
Building Distributed Applications that Work
Ollama is software for downloading, running, and serving large language models locally.
- How it is used
- Hosts large language models locally for an Aspire-based chat application without relying on an external model server.
- How deep it goes
- Describes local hosting, Docker execution, and Aspire toolkit support, but gives no setup or operational steps.
- Examples
- Uses a locally hosted ChatGPT-style assistant and Meta’s Llama as examples.
- What you can do after
- Understand how to host an LLM for a .NET chat application using Ollama and Aspire.
Read how Building Distributed Applications that Work explains Ollama →
Also covered in
- Build a Multi-Agent System (from Scratch)In early access (MEAP)
- Hugging Face in Action
- Knowledge Graphs and LLMs in Action
- Just Use Postgres!
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- large language modelA large language model is a machine-learning model trained on extensive text data to process and generate natural language.
- LlamaLlama is a family of large language models developed by Meta.
- privacyPrivacy is the protection of information about people or other sensitive subjects from inappropriate collection, access, use, or disclosure, including control over how it is handled.
Related concepts
- large language modeluses · A large language model is a machine-learning model trained on extensive text data to process and generate natural language.
- execution environmentkind of · An execution environment is the computational context in which a program runs, including its supporting software, available resources, and relevant system state or configuration.
- session historyuses · Session history is an ordered record of messages or other activity associated with a particular session.
Questions about Ollama
What is Ollama?
Ollama is software for downloading, managing, running, and serving language models locally.
Which Manning books explain Ollama?
Build Applications with Local AI Models on a Mac, Vibe Engineering, Domain-Specific Small Language Models, Server-Side WebAssembly, Build a Large Language Model (From Scratch), Building Specialized AI Systems, Spring AI in Action, AI-Powered Developer: Build software with ChatGPT and Copilot and 2 more.
What should I know before learning Ollama?
Manning books that teach Ollama usually introduce language model, large language model, Llama and privacy first.
What is Ollama a kind of?
Ollama is a kind of local model runner. Software that manages and runs machine-learning models on a local computer.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.









