liveBook · Manning

Token tree

As explained in Learn Rust in a Month of Lunches

A token tree is a syntactic token or delimited group of tokens that can be matched and repeated by a Rust macro.

A token tree is a token or a delimited group of tokens that a Rust macro can match. The `tt` fragment lets a macro accept flexible input without describing its precise expression or type. section 22.3

Token-tree patterns become especially useful with repetition:

  • `$arg:tt` matches one token-tree argument. section 22.3
  • `$($arg:tt)*` matches zero, one, or many token trees. section 22.3
  • `+` requires one or more repetitions, while `?` makes a fragment optional. section 22.3

Forwarding arbitrary arguments

`write!` captures a destination expression and any number of token trees, then forwards those token trees to `format_args!`. This pattern explains why formatting macros accept varied arguments.

macro_rules! todo {
    () => (panic!("not yet implemented"));
    ($($arg:tt)+) => (panic!("not yet implemented: {}", $crate::format_args!($($arg)+)));
}

`todo!` contrasts an empty pattern with a repeated token-tree pattern, showing how a macro can support both no arguments and formatted arguments. section 22.3

How Learn Rust in a Month of Lunches covers it

How deep it goes
Defines `tt` and demonstrates repetition, but stops short of compiler internals behind built-in macros.
Examples
Uses `write!`, `format_args!`, and `todo!` to show single, repeated, empty, and formatted token-tree inputs.
What you can do after
Evaluate macro definitions to infer whether they accept one, many, optional, or formatted inputs.
How it connects
Connects token trees with macro_rules!, metavariables, repetition, formatting macros, and declarative macro matching.

Also explained in

Compare how 3 books explain token tree →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Learn Rust in a Month of Lunches. The section links point to the parts of the book it draws on.