Ollama is a lightweight runtime for running large language models locally. Its CLI downloads models and makes inference available without specialized server infrastructure. In a Wasm application, it can sit behind a custom provider interface when a ready-made wasi:nn integration is unavailable. section 6.6.2
A practical SmartCMS integration proceeds in layers:
- Install Ollama locally and pull the small gurubot/tinystories-656k-q8 model. The model continues a story from a supplied opening prompt and is chosen for minimal resource requirements, although generated stories are not always coherent. section 6.6.2
- Add an Ollama provider that exposes a custom WIT generate interface. A request supplies a prompt and optional images; a response includes generated text, completion state, model metadata, and optional timing and token counts. section 6.6.2
- Import the custom interface into the component world and call generate from the /api/generate route. Combine the prompt with the response, then store the resulting story under an incrementing generated name. section 6.6.2
- Build the provider and application, then deploy the application manifest so the component links to the Ollama capability provider. section 6.6.2
Generate and save a story
For a prompt of "Once upon a time", create a generate request with no images, read the response text, append it to the prompt, and save it in the default key-value bucket. The HTTP response returns both the generated story name and story content. section 6.6.2
Ollama is not presented as a standardized interface: its integration demonstrates how a custom WIT contract can connect a Wasm component to an external LLM backend. This avoids building a provider around ONNX and wasi:nn, while leaving the application dependent on the provider and running Ollama service. section 6.6.2
Listing 6.20 ollama.wit interface (comments/documentation omitted)
interface generate {
record request {
prompt: string,
images: option<list<string>>
}
record response {
model: string,
created-at: string,
response: string,
done: bool,
context: option<list<s32>>,
total-duration: option<u64>,
load-duration: option<u64>,
prompt-eval-count: option<u16>,
prompt-eval-duration: option<u64>,
eval-count: option<u16>,
eval-duration: option<u64>,
}
generate: func(req: request) -> result<response, string>;
}
The generate contract shows the boundary between the component and provider: prompt input, optional images, generated response text, and optional inference metrics. section 6.6.2
For Kubernetes, run Ollama as a separate service reachable by the provider component. A single-container deployment uses the official image, exposes port 11434, preloads the TinyStories model, keeps it warm for 12 hours, and mounts storage at /root/.ollama. section 8.4.2
Listing 8.12 Ollama deployment and service
apiVersion: apps/v1
kind: Deployment
metadata:
name: ollama
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: ollama
template:
metadata:
labels:
app: ollama
spec:
containers:
- name: ollama
image: ollama/ollama:latest
ports:
- containerPort: 11434
env:
- name: PRELOAD_MODELS #1
value: "gurubot/tinystories-656k-q8" #1
- name: OLLAMA_KEEP_ALIVE
value: "12h"
volumeMounts:
- name: ollama-storage
mountPath: /root/.ollama
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "for model in
➥$PRELOAD_MODELS; do ollama run $model \"\"; done"]
volumes:
- name: ollama-storage
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: ollama #2
namespace: default
spec:
selector:
app: ollama
ports:
- port: 11434
targetPort: 11434 #3
protocol: TCP
The deployment manifest makes the LLM backend explicit: one Ollama replica, a preloaded model, persistent runtime storage, and an in-cluster service endpoint. section 8.4.2
