A Modelfile packages a converted custom model for Ollama and defines how it will be deployed and served locally for inference. It bundles the model reference with template and generation settings so the local server can run it as a single package. section 12.2 section 12.7
Typical contents include:
- the GGUF model name in a FROM line section 12.2.1
- an optional TEMPLATE for the system prompt format section 12.2.1
- PARAMETER entries for context length and stop words section 12.2.1
Minimal deployment flow
After converting a custom model to GGUF, create a Modelfile in the same directory, then run ollama create to register the model and ollama run to chat with it. section 12.2.1 section 12.2
FROM starcoder2-3b_instruct.gguf
PARAMETER num_ctx 4096
TEMPLATE """<|endoftext|>You are an exceptionally intelligent coding assistant that consistently delivers accurate and reliable responses to user instructions.
### Instruction
{{ .Prompt }}
### Response
"""
PARAMETER stop "###"
PARAMETER stop "<|endoftext|>"
PARAMETER stop "<file_sep>"
Shows a concrete Modelfile with FROM, TEMPLATE, context length, and multiple stop tokens. section 12.2.1
cd ./gguf_model ollama create starcoder2-3b-instruct-new -f Modelfile
Shows the create command that turns the Modelfile into a local Ollama model. section 12.2.1
ollama run starcoder2-3b-instruct-new:latest
Shows the resulting run command used to start chatting with the created model. section 12.2.1
