liveBook · Manning

Modelfile

As explained in Domain-Specific Small Language Models

An Ollama configuration file that specifies how to package and serve a converted model, including template and generation parameters.

A Modelfile defines how Ollama packages and serves a model for local inference. For a custom model, create it alongside the converted GGUF file. section 12.2.1

The configuration specifies:

FROM starcoder2-3b_instruct.gguf

PARAMETER num_ctx 4096
TEMPLATE """<|endoftext|>You are an exceptionally intelligent coding assistant that consistently delivers accurate and reliable responses to user instructions.
### Instruction
{{ .Prompt }}
### Response
"""

PARAMETER stop "###"
PARAMETER stop "<|endoftext|>"
PARAMETER stop "<file_sep>"

Example configuration for a code-generation model: it sets a 4096-token context, an instruction-response template, and stop sequences. section 12.2.1

How Domain-Specific Small Language Models covers it

How deep it goes
Defines the FROM, TEMPLATE, and PARAMETER directives and their roles without detailing their implementation.
Examples
Uses a StarCoder2 code-generation model with a 4096-token context, an instruction template, and stop sequences.
What it assumes
Requires a converted GGUF model and a running local Ollama instance; custom models are converted with llama.cpp’s script.
What you can do after
Configure a converted GGUF model for Ollama by specifying its template and generation parameters.
How it connects
Connects Ollama deployment with GGUF models, templates, and text-generation parameters.

Also explained in

Compare how 2 books explain Modelfile →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Domain-Specific Small Language Models. The section links point to the parts of the book it draws on.