A random forest combines predictions from multiple decision trees, using randomized data and feature selection to produce a robust ensemble model. Here it serves two practical roles: measuring row similarity for outlier detection and forecasting time-series values from engineered historical features. section 7.12 section 17.2.6
Two implementations illustrate different uses:
- For outlier detection, pass each row through every tree and compare the leaf nodes reached. Rows sharing leaves with few other records receive higher outlier scores. section 7.12
- For forecasting, create lag features that represent recent, weekly, monthly, or annual history, then fit a random-forest regressor and compare predictions with observed values. section 17.2.6
Forecasting seasonal values
A regression forest can predict a daily series using lags of 1, 7, 14, 28, 30, 360, and 364 days. Weekly and annual patterns become available as features; predictions are then assessed with mean absolute percentage error. The approach performs reasonably on repeating patterns but cannot extrapolate an upward trend when future values differ from all training values. section 17.2.6
Listing 7.16 Distance metric learning outlier detection
from sklearn.ensemble import RandomForestClassifier
from collections import Counter
class DMLOutlierDetection:
def __init__(self):
pass
def fit_predict(self, df):
real_df = df.copy()
real_df['Real'] = True
synth_df = pd.DataFrame() #1
for col_name in df.columns:
mean = df[col_name].mean()
stddev = df[col_name].std()
synth_df[col_name] = np.random.normal(loc=mean,
scale=stddev, size=len(df))
synth_df['Real'] = False
train_df = pd.concat([real_df, synth_df])
clf = RandomForestClassifier(max_depth=5)
clf.fit(train_df.drop(columns=['Real']), train_df['Real'])
r = clf.apply(df) #2
scores = [0]*len(df) #3
for tree_idx in range(len(r[0])): #4
c = Counter(r[:, tree_idx]) #5
for record_idx in range(len(df)): #6
node_idx = r[record_idx, tree_idx]
node_count = c[node_idx]
scores[record_idx] += len(df) - node_count
return scores
df = create_four_clusters_test_data()
df = pd.DataFrame(RobustScaler().fit_transform(df), columns=df.columns)
clf = DMLOutlierDetection()
df['Scores'] = clf.fit_predict(df)
Shows a scikit-learn implementation that labels real and synthetic rows, fits a classifier, applies the forest to the original data, and counts shared leaf nodes. section 7.12
Listing 17.8 Forecasting values using a Random Forest
from sklearn.ensemble import RandomForestRegressor df['Lag 1'] = df['Values'].shift(1) #1 df['Lag 7'] = df['Values'].shift(7) #2 df['Lag 14'] = df['Values'].shift(14) df['Lag 28'] = df['Values'].shift(28) df['Lag 30'] = df['Values'].shift(30) #3 df['Lag 360'] = df['Values'].shift(360) #4 df['Lag 364'] = df['Values'].shift(364) x_train = df.drop(columns=['timestamp','Values'])[:1000] #5 y_train = df['Values'][:1000] x_train = x_train.dropna() y_train = y_train.loc[x_train.index] x_test = df.drop(columns=[ 'timestamp','Values'])[1000:] y_test = df['Values'][1000:] regr = RandomForestRegressor() #6 regr.fit(x_train, y_train) y_pred = regr.predict(x_test)
Shows a scikit-learn regression implementation using lagged values, a train/test split, and RandomForestRegressor predictions. section 17.2.6
