A policy turns observed information into an action. In a sequential decision system, it can choose a medication dose from a patient’s pain history, or an investment allocation from available market information. section 7.5.2 section 12.4.1
Write a policy as π(·): a function that receives observed outcomes or context and returns an action. When action selection includes randomness, the policy is a stochastic intervention. It changes the action mechanism while retaining dependence on variables supplied to the policy. section 12.4 section 7.5.2
Causal analysis makes three operational features explicit:
- A policy can replace the action’s original incoming causes, while context-dependent selection creates a new influence from context to action. section 12.4
- A policy can use a complete history rather than the current state, as bandit algorithms do when selecting actions from past rewards and actions. section 12.4.1
- A policy can introduce confounding: it may remove one backdoor path while leaving another through a context variable that requires adjustment. section 12.4.2
- In an MDP, a policy selects actions from state; implementation becomes difficult when states are high-dimensional objects. section 12.5.4
Medication policy
Imagine a doctor deciding a patient’s next dose from recorded pain levels and previous dosages. The utility function balances pain relief against overdose and addiction risk, so the policy must use historical information rather than pain alone. section 12.4.1
Figure 12.23 The dashed lines show edges modulated by the policy. The policy breaks the influence of the confounder Z like an ideal intervention, but dependence on C remains through the policy.
The causal graph shows which action influences are removed by a policy and which context-dependent influence remains. section 12.4
