A pseudo-inverse, or Moore–Penrose pseudo-inverse, generalizes the ordinary matrix inverse. It remains defined when an inverse is undefined, including cases with too few samples or highly similar regressors, while matching the ordinary inverse when that inverse exists. section 5.1.1
Linear regression for each action
For each action, estimate coefficients with β = pinv(XᵀX)Xᵀy. In NumPy, np.linalg.pinv(x.T @ x) @ x.T @ y provides a more robust alternative to np.linalg.inv() when fitting models from logged contexts and rewards. section 5.1.1
Listing 5.4 Build a model for each action
def build_models(num_features, samples_y, samples_x):
betas = []
for y, x in zip(samples_y, samples_x):
y = np.array(y) #1
x = np.array(x) #1
if len(y) > 0: #2
beta = np.linalg.pinv(x.T@x) @ x.T@y #3
else:
beta = np.zeros(shape=(num_features,))
betas.append(beta) #4
return betas
Inspect the complete build_models implementation, including empty-action handling and coefficient collection. section 5.1.1
