liveBook · Manning

Model-to-code interface

As explained in Agent Design Patterns

A code–AI interface is a boundary where program code exchanges model outputs that must conform to machine-readable expectations.

A model-to-code interface is the point where an agent’s orchestration code receives a model response and expects to use it in a specified format. A response that seems understandable to a person can still break the agent: code cannot reliably search free-form prose for the intended answer when it expects a particular structure. section 5.0 section 3.3.4

A math-answer handoff

Consider an agent that asks a language model to reason through a student’s math problem and conclude with a final answer. If the agent’s code expects that answer in a specified form, a differently formatted reply creates a handoff problem even when a person could find the answer in the response. Probabilistic sampling makes exact repetition of the requested form uncertain. section 5.0

Two responses to this boundary problem are:

  • Constrain generation with a supported grammar or format specification, such as a JSON grammar or schema, when code needs parseable output. section 3.3.4
  • Separate structured extraction from less constrained reasoning: pass unstructured content to an extraction errand that produces the structured result. section 3.3.4
  • Use repeated sampling as a reliability pattern when stochastic output causes failures, while recognizing that another sample alone does not define a parseable interface. section 5.0
Listing 3.6 structured_extraction_errand.py: Errand for JSON extraction
class ExtractToJSONErrand(Errand):
    def __init__(self, client: Client):
        self.system_prompt = "You are a helpful assistant that extracts 
structured data from a free-form textual response in JSON format. Output 
the extracted data as a JSON object. Do not include any other text in your 
response."
 
        self.extraction_call = LMCall(
            client,
            [
                {
                    "role": "system",
                    "content": self.system_prompt,
                }
            ],
        )
 
    async def __call__(self, input: str) -> str:
        extraction_response: str = (
            await self.extraction_call(
                input,
                config={
                    "response_format": {
                        "type": "json_object"
                    }
                },
            )
        )
        return json.loads(extraction_response)
 

The JSON extraction errand illustrates both sides of the handoff: a prompt requests only a JSON object, while the model-call configuration sets `response_format` to `json_object`. The endpoint’s format control supplements the prompt rather than relying on the instruction alone. section 3.3.4

How Agent Design Patterns covers it

How deep it goes
Explains formatting failures and a configured JSON extraction errand; does not establish a general interface contract or recovery procedure.
Examples
Uses a student math-answer agent and a JSON extraction errand configured with `response_format`.
What it assumes
Assumes familiarity with agent code calling language models; introduces the risk posed by stochastic, incorrectly formatted responses.
What you can do after
Apply it via a structured extraction errand to request JSON output at an agent’s model-to-code handoff.
How it connects
Connects probabilistic sampling and resampling with controlled generation, structured extraction errands, and agent orchestration.

This explainer was generated by Manning's knowledge graph from the text of Agent Design Patterns. The section links point to the parts of the book it draws on.