A resource allocation problem assigns a limited budget across competing projects whose costs, returns, and chances of success differ. Because outcomes are uncertain and spending affects what remains, the decision can be modeled as a Markov decision process (MDP). section 4.4
Represent each state with the remaining budget and a status for each project. Actions either stop or allocate funds to an eligible project; transitions capture possible outcomes, and rewards balance returns against allocation costs and losses. Dynamic programming evaluates and improves a policy to find allocations with higher expected value. section 4.4
Startup project portfolio
A startup chooses among a predictable product improvement, a high-risk innovation, a very safe maintenance project, and a moderate-risk market expansion. Each competes for a limited budget, and results arrive only after funds are committed. section 4.4
Listing 4.1 Defining Problem Class
class ResourceAllocationMDP:
n_projects: int = 4 #A
max_budget: int = 10 #B
max_allocation: int = 5 #C
discount_factor: float = 0.9 #D
def __post_init__(self):
self.project_names = [f"Project_{chr(65+i)}" for i in range(self.n_projects)] #E
self.project_params = { #F
0: {"base_return": 2.1, "volatility": 0.3, "success_prob": 0.7},
1: {"base_return": 2.9, "volatility": 0.5, "success_prob": 0.5},
2: {"base_return": 1.8, "volatility": 0.2, "success_prob": 0.8},
3: {"base_return": 2.3, "volatility": 0.4, "success_prob": 0.6},
}
Inspect the Python class defining the number of projects, budget limits, allocation limits, discount factor, and project parameters. section 4.4
