A delivery agent choosing its next customer needs a defined set of possible choices. That set is its action space: the actions the agent can submit to an environment at each decision step. The space should represent real decision options and respect domain constraints. section 3.1 section 3.5
Actions can be discrete, such as choosing a destination, or continuous, such as adjusting a quantity or speed. An action can also combine several fields. The environment uses the submitted action in its transition logic to update the state and determine consequences and rewards. section 3.1 section 3.4
Resource allocation
An action pairs a project ID with an allocation amount; (-1, 0) means stop allocating. The overall list includes positive allocations up to the configured maximum and total budget. At a particular state, an allocation is valid only if it fits the remaining budget and the project has not already been attempted. section 4.4
Listing 4.3 Define Action Space
def create_action_space(mdp: ResourceAllocationMDP) -> list[tuple]:
actions = [(-1, 0)] #A
for project_id in range(mdp.n_projects): #B
for allocation in range(1, min(mdp.max_allocation + 1, #C
mdp.max_budget + 1)):
actions.append((project_id, allocation))
return actions
The listing constructs the allocation action list, including the special do-nothing action. section 4.4
Packing a trailer
A placement action is a tuple of container index, x, y, z, and orientation index. A proposed placement must fit within the container and avoid occupied space. A valid placement updates the grid; an invalid one receives a penalty, and repeated failures can cause the item to be skipped. section 3.4
Action-space design also affects which methods fit. A deep Q-network works best with a moderate-sized discrete space because it outputs one value per action; it cannot directly handle continuous actions and struggles with enormous spaces. For portfolio management, choosing among eight named allocation templates instead of raw continuous weights makes decisions readable and supports a categorical-policy PPO. The templates are designed to respect position caps. section 8.5 section 12.4
The decision step need not correspond to a whole business outcome. In language generation, the action space is the vocabulary of possible next tokens, not all possible sentences. A response emerges through repeated token choices. section 10.2



