liveBook · Manning

Structured tool argument extraction

As explained in Rearchitecting LLMs

Structured tool argument extraction is the example application's task of converting free-form user intent into validated tool names and typed parameters.

Structured tool argument extraction converts free-form user intent into validated JSON specifying a tool name and typed parameters. In an agent workflow, confirmed tool executions and schema-valid outputs provide examples for specializing a smaller model. section 12.2

A specialization workflow:

  • Sample agent requests, redact or hash sensitive inputs before storing them, then filter for schema validity and confirmed downstream execution. section 12.2
  • Split the resulting pairs into training and held-out evaluation sets; use the task data to guide depth pruning, hardware-aligned width pruning, and selective attention removal. section 12.2 section 12.3
  • Recover capability with sequence-level distillation on the teacher’s hard completions; check provider terms before training on API outputs. section 12.3
{"input": "Book a table for two tomorrow at 8 pm",
 "target": {"tool_name": "book_table",
            "parameters": {"party_size": 2, "time": "20:00"}},
 "confirmed": true}

A table-booking request is paired with a confirmed tool name and typed parameters, including party size and time. section 12.2

How Rearchitecting LLMs covers it

How deep it goes
Details dataset construction, structural pruning, and recovery training, while deferring efficient production serving.
Examples
Uses a table-booking request and a Llama-3.2-3B specialization measured against a frontier API.
What it assumes
Assumes access to agent requests, output schemas, execution confirmation, and teacher completions for recovery training.
What you can do after
Build a task-specialized SLM by filtering agent traffic, pruning for the task, and recovering accuracy through hard-completion training.
How it connects
Connects tool schemas and JSON validation with model pruning, KV-cache reduction, and sequence-level distillation.

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.