liveBook · Manning

AI agent

As explained in LLM Evaluation and Alignment, The Foundational Ideas

A decision-maker that interacts with an environment by taking actions and receiving feedback to achieve a goal.

An AI agent is a decision-maker that interacts with an environment to achieve a goal by taking actions and receiving feedback as rewards. A coding agent may browse documentation, edit files, and run tests. section 1.1.1

Evaluate an agent by checking whether it:

  • Selects the correct tool or API at each step. section 1.1.1
  • Uses the tool appropriately while pursuing its goal. section 1.1.1
  • Produces outcomes that can be tested objectively, such as passing code tests. section 1.1.1

Coding agent

A coding agent edits a file and runs its tests. Passing tests provide a reward signal; failing tests provide no reward and expose a verifiable error. section 1.1.1

Listing 1.1 Verifiable rewards example (HumanEval)
def has_close_elements(numbers: List[float], threshold: float) -> bool:
    “””Check if in given list of numbers, are any two numbers
    closer to each other than given threshold.
    >>> has_close_elements([1.0, 2.0, 3.0], 0.5)
    False
    >>> has_close_elements([1.0, 2.8, 3.0, 4.0, 5.0, 2.0], 0.3)
    True

Use the HumanEval example to see how executable tests turn an agent-produced coding task into a reproducible reward check. section 1.1.1

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines actions, environments, goals, feedback, and rewards, then stops short of describing agent architecture.
Examples
Uses coding agents, documentation browsing, file editing, API calls, and executable tests as examples.
What it assumes
Assumes no reinforcement-learning background and teaches the agent–environment interaction through coding examples.
What you can do after
Evaluate agent behavior by verifying tool choices, tool use, and resulting task outcomes.
How it connects
Connects agents to environments, actions, rewards, feedback, tools, APIs, and verifiable tasks.

Also explained in

Compare how 46 books explain AI agent →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.