liveBook · Manning

Action space

As explained in AI Model Evaluation

An action space is the set of actions available to a policy or decision-making system in a given context.

An action space is the set of actions available to a policy at a decision point. In counterfactual evaluation, its size determines how much useful evidence logged data can provide: choosing among 10 candidates differs sharply from choosing among millions. section 5.3.2 section 5.3

Treat the action space as part of the decision record, not as an abstract universe of possibilities:

  • Record the feasible candidate set considered at decision time, alongside the selected action and its selection probability. section 5.3
  • Check whether the logged policy explored enough alternatives and overlapped with the actions a new policy would choose. section 5.3.2
  • Expect instability when the action space is large, exploration is scarce, or effective sample size is only a small fraction of total logs. section 5.3.2
  • Decompose agentic decisions into stages—model, tool, retrieval source, clarification, escalation—rather than evaluating the entire chain as one action. section 5.9.1

Search ranking

For a wireless-earbuds query, the action space for position one contains four considered products. Logging their exact selection probabilities—0.52, 0.28, 0.13, and 0.07—along with context and clicks supports the question, “What if Sony earbuds had ranked first?” section 5.5.5

Listing 5.3 Search log event example for a ecommerce website
{
  "session_id": "sess_20241125_210315_user_78234",
  "timestamp": "2024-11-25T21:03:15.342Z",
  "user_id": "user_78234",
  "context": {
    "page": "search_results",
    "device": "desktop",
    "query": "wireless earbuds",
    "time_of_day": "evening"
  },
  "available_actions": [
    {
      "action_id": "rank_position_1",
      "product_id": "prod_84729",
      "product_name": "AirPods Pro Max",
      "price": 249.99,
      "avg_rating": 4.6,
      "relevance_score": 0.94
    },
    {
      "action_id": "rank_position_1",
      "product_id": "prod_52846", 
      "product_name": "Sony WF-1000XM4",
      "price": 199.99,
      "avg_rating": 4.4,
      "relevance_score": 0.91
    },
    {
      "action_id": "rank_position_1",
      "product_id": "prod_73958",
      "product_name": "Bose QuietComfort Earbuds",
      "price": 179.99,
      "avg_rating": 4.2,
      "relevance_score": 0.88
    },
    {
      "action_id": "rank_position_1",
      "product_id": "prod_91647",
      "product_name": "Samsung Galaxy Buds Pro",
      "price": 149.99,
      "avg_rating": 4.3,
      "relevance_score": 0.85
    }
  ],
  "action_probabilities": {
    "prod_84729": 0.52,
    "prod_52846": 0.28,
    "prod_73958": 0.13,
    "prod_91647": 0.07
  },
  "selected_action": {
    "product_id": "prod_84729",
    "selection_probability": 0.52,
    "model_version": "search_ranking_v3.1.8"
  },
  "outcome": {
    "clicked": true,
    "click_position": 1,
    "time_to_click_seconds": 6.8,
    "added_to_cart": false,
    "viewed_alternatives": ["prod_52846", "prod_73958"]
  }
}

Inspect the complete search decision record: context, available products, ranking action, probabilities, and outcome fields. section 5.5.5

How AI Model Evaluation covers it

How deep it goes
Explains candidate logging, overlap, exploration, sample-size warnings, and staged agent decisions, but not formal action-space mathematics.
Examples
Uses movie recommendation, search ranking, wireless-earbud candidates, and multi-step agent decisions as concrete cases.
What it assumes
Assumes policies make production decisions and teaches from zero which decision fields make alternatives observable.
What you can do after
Evaluate counterfactual policies by logging candidates, selected actions, probabilities, context, versions, and outcomes.
How it connects
Connects to policies, counterfactual evaluation, exploration, candidate sets, propensities, ranking, and agentic systems.

Also explained in

Compare how 4 books explain action space →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.