A draft model proposes candidate tokens autoregressively; a target model verifies them together in one forward pass. The target accepts matching tokens, corrects the first disagreement, and drafting resumes from there. section 13.6.1
Predictable agent output
Tags, JSON keys, whitespace, and known tool names are predictable structural tokens a draft model can propose; the target can focus its corrections on argument values. section 13.6.1
Figure 13.5 One iteration of speculative decoding with K = 5. The draft model S proposes five candidate tokens one at a time; the target model T verifies all five in a single forward pass. Tokens 1–3 are accepted. The first disagreement occurs at token 4, so tokens 4 and 5 are discarded and T writes one token of its own in position 4. In the next iteration, S resumes drafting from T's token. The tokens are illustrative digits of a ZIP code, and the disagreement at token 4 is constructed for this example.

Follow a five-token ZIP-code example: the target accepts three tokens, replaces the fourth after a disagreement, and the draft resumes from that correction. section 13.6.1

