liveBook · Manning

Chat template

As explained in Rearchitecting LLMs

A chat template is a formatting specification that structures messages and other content into a model's expected conversational input format.

A chat template formats messages and other content into the conversational structure expected by a model. Applying the template keeps training examples aligned with the format used at inference. section 7.3.3 section 13.2.2

A practical workflow uses the tokenizer’s chat template to:

  • Build messages with roles such as system, user, and assistant, placing instructions, input, and target output in the appropriate messages. section 7.3.3
  • Serialize the messages with tokenizer.apply_chat_template; for tool-call training, construct both the full example and the prompt so training targets only the completion. section 13.2.2

Clinical extraction

A system message containing “Extract:” and a user message containing a clinical note are paired with the assistant’s validated JSON output. The template formats this exchange for training. section 7.3.3

Figure 13.2 Split of one training example after applying the chat template. The prompt contains everything the model receives as input: the system instructions, the available tools, the user query, and the opening assistant tags. The completion is only what the model must generate: the tool call, wrapped in the template's response tags.

The example shows how a tool-calling exchange becomes a prompt containing instructions, tools, and the user query, plus a completion containing the generated tool call. section 13.2.2

How Rearchitecting LLMs covers it

How deep it goes
Demonstrates role-based serialization and prompt-completion construction through tokenizer calls.
Examples
Uses clinical-note-to-JSON training examples and Qwen3/xLAM tool-call examples.
What it assumes
Assumes a model tokenizer and dataset examples; explains system, user, and assistant roles.
What you can do after
Apply it via tokenizer.apply_chat_template to serialize messages and separate prompts from target completions.
How it connects
Connects tokenizer configuration, supervised fine-tuning, and tool calling.

Also explained in

Compare how 2 books explain chat template →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.