A constant feature has the same value for every observation, so it provides no variation from which a machine-learning model can learn. In a tabular dataset, remove it during exploration rather than carrying it through later numerical processing. section 2.2.1
Use these checks to distinguish useless columns from merely low-variation columns:
- For numeric features, treat variance approximately equal to zero as a warning sign. section 2.2.1
- For categorical features, investigate a majority class above 99.9%; this indicates a quasi-constant column. section 2.2.1
- Drop constant and quasi-constant columns because constant values add processing burden, while tiny nonconstant parts can encourage overfitting. section 2.2.1
Listing 2.2 Dropping zero variance features
import pandas as pd
from sklearn.preprocessing import OrdinalEncoder
from sklearn.feature_selection import VarianceThreshold
data = pd.DataFrame({"feature_1":['A' for i in range(15)],
"feature_2":['B' if i%2==0 else 'C'
for i in range(15)],
"feature_3":[i**2 for i in range(15)]})
ord_enc = OrdinalEncoder() #1
data[data.columns] = ord_enc.fit_transform(data)
var_threshold = VarianceThreshold(threshold=0) #2
clean_data = var_threshold.fit_transform(data)
print(var_threshold.variances_) #3
print(clean_data.shape)
Encode categorical values numerically, apply scikit-learn's zero-variance threshold, inspect the computed variances, and retain only varying columns. section 2.2.1
