A regression coefficient is the weight assigned to a numeric feature in a linear regression model. The model multiplies every feature value by its coefficient and sums those contributions to produce a prediction. During fitting, it chooses coefficients that make predictions approximate the target by minimizing residual sum of squares. section 4.3.1
The sign and size of a coefficient help explain a feature’s contribution: a positive coefficient increases the prediction as its feature value increases, while a negative coefficient decreases it. Inspecting coefficients alongside their features can support explanations to stakeholders, regulatory review, and checks against domain knowledge. section 4.3.1
Coefficient interpretation depends on the data preparation and model design:
- Categorical features must first be numeric, such as through one-hot encoding, so they can receive coefficients. section 4.3.1
- Missing values and outliers need attention because they affect the multiplications used in training and prediction. section 4.3.1
- Transformations and interaction features can give a linear model more flexibility, but adding many features can increase variance and overfitting. section 4.3.2
- Multicollinearity can make coefficient-based explanations conceptually misleading. section 4.4
Regularization changes how coefficients are selected by penalizing their total magnitude. L2 regularization generally shrinks coefficients, whereas L1 regularization can push many coefficients to zero, making their associated features irrelevant and supporting feature selection. Both approaches constrain model complexity to improve generalization when many features are available. section 4.3.2 section 4.4
Diagnosing an extreme prediction
For an individual prediction, multiply each feature value by its coefficient and inspect the resulting contributions. In a rental-listing baseline, a high bedroom value created an unusually large positive contribution, while a high minimum-nights value created a large negative contribution. Such cases reveal how unbounded linear contributions can produce outliers and suggest nonlinearities or interactions as possible remedies. section 7.2.3
Listing 7.15 Inspection of the coefficients
def report_case(model, data, feature_names, case_no):
case_values = data[case_no] #1
coef_values = case_values * model.coef_ #2
for feature_name, value, coef_value in zip(
feature_names, case_values, coef_values):
print(f"{feature_name:50s}" +
f"({value:10.2f}) : " +
f"{coef_value:+0.2f}") #3
print("-" * 80)
print(" "*66 + f"{np.sum(coef_values):+0.2f}") #4
This inspection routine computes each feature value times its coefficient, then prints the contributions and their sum for a selected prediction. section 7.2.3
