Ollama
Ollama is software for downloading, running, and serving language models locally.
Build Applications with Local AI Models on a Mac
Ollama is a tool for downloading, running, and serving language models locally.
- How it is used
- Runs and serves local LLMs for a private Mac-based assistant that connects model responses to Python and a web interface.
- How deep it goes
- Explains the runtime with an analogy and request flow, then gives installation, service-startup, verification, and connection-error checks without model internals.
- Examples
- Uses a music-player analogy, Homebrew installation, a localhost service check, and a Python connection-refused error.
- What you can do after
- Operate/configure it to install Ollama, start its service, run local models, and diagnose a failed localhost connection.
Read how Build Applications with Local AI Models on a Mac explains Ollama →
Domain-Specific Small Language Models
A local model runtime and serving tool that downloads, runs, and exposes quantized language models through an API and chat interface.
- How it is used
- Runs personal or productivity-oriented language-model assistants locally, including fully offline workflows.
- How deep it goes
- Explains installation, model retrieval, quantization, hardware needs, APIs, packaging, and privacy caveats without detailing server internals.
- Examples
- Demonstrates DeepSeek-R1 and Mistral commands, Hugging Face model retrieval, model listing, deletion, and a curl chat request.
- What you can do after
- Operate local quantized models, expose them through an API, and assess hardware and privacy constraints.
Read how Domain-Specific Small Language Models explains Ollama →
Server-Side WebAssembly
Ollama is a local machine-learning runtime and command-line tool for downloading and running large language models.
- How it is used
- Provides local or cluster-hosted LLM inference for SmartCMS story generation through a custom Wasm provider.
- How deep it goes
- Explains installation, model pulling, WIT contract integration, component calls, storage, and Kubernetes deployment without implementing the provider runtime.
- Examples
- Uses a TinyStories continuation model, a /api/generate endpoint, generated-story storage, and an Ollama Kubernetes Deployment and Service.
- What you can do after
- Apply it via Ollama, a custom provider, and Kubernetes manifests to generate and persist SmartCMS stories.
Build a Large Language Model (From Scratch)
A local application and serving wrapper for running large language models for inference through a command line or API.
- How it is used
- Supports local LLM inference and automated evaluation of instruction-fine-tuned model responses.
- How deep it goes
- Explains startup, model execution, API requests, and scoring code, while stopping short of training or fine-tuning internals.
- Examples
- Uses ollama run llama3, a llama prompt, Python API queries, and automated scores over 110 test entries.
- What you can do after
- Operate/configure a local Llama 3 service and query it from Python for text generation or response scoring.
Read how Build a Large Language Model (From Scratch) explains Ollama →
Spring AI in Action
Ollama is a tool for running and serving open-source language models locally on user hardware.
- How it is used
- Runs open-source language models locally for development and Spring AI applications.
- How deep it goes
- Explains installation, model selection, dependency setup, configuration, and hardware trade-offs without detailing model internals.
- Examples
- Uses Gemma 2B, Gemma 7B, Mistral 7B, Ollama commands, and the /api/tags endpoint.
- What you can do after
- Operate/configure Ollama locally and connect a selected model to Spring AI without cloud credentials.
AI-Powered Developer: Build software with ChatGPT and Copilot
Software for running and managing large language models locally, including through Docker-based deployments.
- How it is used
- Deploys Llama 2 locally for offline experimentation and comparison with managed LLM output.
- How deep it goes
- Explains Docker startup, model downloading, prompt refinement, and output checking, but does not describe Ollama internals.
- Examples
- Uses Docker commands, model self-description, pure-Python standard-deviation code, formula explanations, and degrees-of-freedom questions.
- What you can do after
- Operate Ollama in Docker, run Llama 2 locally, and evaluate or refine its answers against a known-correct baseline.
Read how AI-Powered Developer: Build software with ChatGPT and Copilot explains Ollama →
Data-Driven Cybersecurity
Ollama is a tool for running and interacting with large language models locally through command-line and API interfaces.
- How it is used
- Deploys local language models for privacy-conscious cybersecurity analysis and integration into custom applications or automated workflows.
- How deep it goes
- Explains local interaction, privacy, performance, customization, API integration, and command-line use, but provides no Ollama-specific code or installation procedure.
- Examples
- Demonstrates cybersecurity insight generation, local handling of sensitive data, and comparison with GUI-based LM Studio workflows.
- What you can do after
- Operate/configure it for local model use, API-connected applications, customized tasks, and privacy-sensitive cybersecurity workflows.
Building Distributed Applications that Work
Ollama is software for downloading, running, and serving large language models locally.
- How it is used
- Hosts large language models locally for an Aspire-based chat application without relying on an external model server.
- How deep it goes
- Describes local hosting, Docker execution, and Aspire toolkit support, but gives no setup or operational steps.
- Examples
- Uses a locally hosted ChatGPT-style assistant and Meta’s Llama as examples.
- What you can do after
- Understand how to host an LLM for a .NET chat application using Ollama and Aspire.
Read how Building Distributed Applications that Work explains Ollama →
Related concepts
- large language modeluses · A large language model is a neural machine-learning model trained on extensive text data to process or generate natural-language sequences.
- execution environmentkind of · An execution environment is the computing context, including hardware, software, configuration, and supporting resources, in which programs run.
- session historyuses · Session history is an ordered record of entries or interactions associated with a particular session.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.