A Modelfile defines how Ollama packages and serves a model for local inference. For a custom model, create it alongside the converted GGUF file. section 12.2.1
The configuration specifies:
- FROM names the GGUF model. section 12.2.1
- TEMPLATE optionally defines a system template. section 12.2.1
- PARAMETER sets generation options, such as context length and stop words. section 12.2.1
FROM starcoder2-3b_instruct.gguf
PARAMETER num_ctx 4096
TEMPLATE """<|endoftext|>You are an exceptionally intelligent coding assistant that consistently delivers accurate and reliable responses to user instructions.
### Instruction
{{ .Prompt }}
### Response
"""
PARAMETER stop "###"
PARAMETER stop "<|endoftext|>"
PARAMETER stop "<file_sep>"
Example configuration for a code-generation model: it sets a 4096-token context, an instruction-response template, and stop sequences. section 12.2.1

