A feature matrix is a rectangular numerical structure where each row represents an observation and each column represents an input feature. Before linear regression uses it, categorical features must be converted to numeric columns, commonly through one-hot encoding. section 4.3.1
Regression inputs
A model multiplies the feature matrix by a column vector of learned coefficients to produce predictions. For example, encoded category columns and transformed numeric columns become inputs to a regression pipeline. Missing values must be imputed before multiplication or addition, and outliers require attention. section 4.3.1
Listing 4.3 Linear regression
from sklearn.linear_model import LinearRegression
from sklearn.metrics import make_scorer, mean_squared_error
from sklearn.model_selection import KFold
from sklearn.model_selection import cross_validate
column_transform = ColumnTransformer(
[('categories', categorical_onehot_encoding, low_card_categorical),
('numeric', numeric_passthrough, continuous)],
remainder='drop',
verbose_feature_names_out=False,
sparse_threshold=0.0) #1
model = LinearRegression() #2
model_pipeline = Pipeline(
[('processing', column_transform),
('modeling', model)] #3
)
cv = KFold(5, shuffle=True, random_state=0) #4
rmse = make_scorer(mean_squared_error,
squared=False) #5
cv_scores = cross_validate(estimator=model_pipeline,
X=data[price_window],
y=target_regression[price_window],
scoring=rmse,
cv=cv,
return_train_score=True,
return_estimator=True) #6
mean_cv = np.mean(cv_scores['test_score'])
std_cv = np.std(cv_scores['test_score'])
fit_time = np.mean(cv_scores['fit_time'])
score_time = np.mean(cv_scores['score_time'])
print(f"{mean_cv:0.3f} ({std_cv:0.3f})",
f"fit: {fit_time:0.2f}",
f"secs pred: {score_time:0.2f} secs") #7
Inspect the regression pipeline to see categorical and numeric columns combined into model-ready features before cross-validation. section 4.3.1
