A chat template is the formatting contract between structured conversation data and a language model’s token stream. A base model simply continues tokens; an instruction-tuned model must instead receive a serialized dialogue whose role labels, delimiters, special tokens, whitespace, and turn order match the format used for training. This makes the template a standard interface between users and instruction-tuned models, and a dependency of later post-training stages. section 4.1 section 4.4
Templates commonly accept a list of message dictionaries with system, user, and assistant roles. A system message appears first and supplies hidden agent instructions or context; user messages carry requests; assistant messages represent model replies. The template adds model-specific markup around each message, verifies user/assistant alternation, and may finish with an empty assistant tag so generation starts in the correct place. section 4.1
From messages to a generation prompt
A conversation can serialize as `<|im_start|>system ... <|im_end|>`, followed by a user turn, then `<|im_start|>assistant`. The final empty assistant header is not content to answer; it cues the model to produce the next assistant tokens. During training, the same structure includes completed assistant turns, while prompt and multiturn masking determine which tokens contribute to learning. section 4.1 section 4.4
{% if messages[0]['role'] == 'system' %}
{% set offset = 1 %} 1
{% else %}
{% set offset = 0 %}
{% endif %}
\hspace{-2pt}{{ bos_token }} \hspace{-8pt}2
{% for message in messages %}
{% set expected_user = loop.index0 % 2 == offset %} \hspace{-8pt}3
{% if (message['role'] == 'user') != expected_user %}
{{ raise_exception(
'Conversation roles must alternate '
~ 'user/assistant/user/assistant/...'
) }}
{% endif %}
{{ '<|im_start|>' + message['role'] + '\n'
+ (message['content'] | trim)
+ '<|im_end|>\n' }} 4
{% endfor %}
{% if add_generation_prompt %}
{{ '<|im_start|>assistant\n' }} \hspace{-8pt}5
{% endif %}
This Jinja template shows the operational pipeline: detect an optional system message, insert a beginning-of-sequence token, enforce alternating roles, wrap content in chat markup, and optionally append an assistant generation prompt. section 4.1
Chat templates also carry structured tool-use interactions into the model context. Tool-calling datasets retain the overall message structure but split model turns into content subsections around tool calls. Structured JSON message representations are converted to raw token streams, with architectures differing on whether tool boundaries use special tokens or structured token-stream formatting. Template playgrounds make these conversions inspectable across models. section 13.5
Implementation choices have direct training and serving consequences:
- Match the template to the target model’s expected special tokens and role markup; similar conversations can have technically distinct serializations. section 4.1 section 13.5
- Preserve the formatting distribution used downstream, since instruction tuning depends on data quality and prompts that match intended tasks. section 4.4
- Decide how multiturn content is masked and whether reasoning-token continuity is retained across tool calls, balancing context retention against serving cost. section 4.4 section 13.5
