liveBook · Manning

Token tree

As explained in Write Powerful Rust Macros

A Rust token tree is a syntactic token or delimited sequence of tokens that can be captured as a broadly permissive macro fragment.

A token tree is a unit within Rust macro input: a TokenStream is a sequence of TokenTrees, and each tree can recursively contain further trees or tokens. Declarative macros capture this broad shape with the `tt` fragment specifier. section 3.6

Use `tt` when a matcher must preserve syntax that is too varied for a narrower fragment such as `ident`, `expr`, or `ty`. Its flexibility is deliberate, but it also accepts so much that mistakes become harder to reject; start with a concrete fragment and widen only when needed. section 2.1.4

Forwarding optional macro input

A `lazy_static!` matcher captures trailing optional input as `$($t:tt)*` and passes it to an internal macro. The token-tree repetition keeps that extra syntax intact without requiring the entry matcher to classify every possible token. section 2.3

Inspecting procedural-macro input

A procedural macro can iterate over its incoming TokenStream, take a TokenTree, and match it as an identifier. This works for a small derive macro, though manually extracting and generating syntax grows burdensome for richer input. section 3.6

Listing 2.20 lazy_static macro entry point, simplified slightly
#[macro_export]
macro_rules! lazy_static {
  ($(#[$attr:meta])* static ref $N:ident : $T:ty =
 $e:expr; $($t:tt)*) => {
    __lazy_static_internal!($(#[$attr])* () static ref $N :
 $T = $e; $($t)*);
    };
    // other arms...
}

Inspect a practical matcher that combines typed fragments with a repeated `tt` tail for additional input. section 2.3

How Write Powerful Rust Macros covers it

How deep it goes
Defines TokenTrees within TokenStreams and demonstrates matcher capture and direct procedural-macro inspection, while deferring complete syntax parsing.
Examples
Uses a `lazy_static!` trailing matcher and a derive macro that extracts a struct or enum name.
What it assumes
Assumes Rust macro syntax and contrasts `tt` with familiar fragments such as `expr`, `ident`, literals, and types.
What you can do after
Apply it via `macro_rules!` matchers or procedural TokenStream iteration when macro input must be captured or inspected.
How it connects
Connects TokenStream, declarative macros, fragment specifiers, procedural macros, syn, quote, and venial.

Also explained in

Compare how 3 books explain token tree →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Write Powerful Rust Macros. The section links point to the parts of the book it draws on.