Prediction time measures how long a trained model or detector takes to produce outputs for input records. It is distinct from fit time: a detector may train slowly but predict quickly, or do the reverse. The metric matters when data must be evaluated rapidly or in large volumes. section 8.7 section 8.0
Estimate prediction time with realistic workload experiments. Duplicate a current dataset to several row counts, run each detector, and compare elapsed times as workload grows. Use the results to weigh accuracy against performance rather than treating any detector as universally best. section 8.7 section 8.0
Interpret prediction-time measurements by checking:
- How runtime changes with the number of rows; constant or approximately linear growth can remain manageable, while steep growth can become limiting. section 8.7
- Whether training and prediction have different bottlenecks; KNN can predict longer than it trains, whereas IF and CBLOF can train longer than they predict. section 8.7
- Whether a faster detector or a smaller ensemble provides an acceptable balance of accuracy, prediction time, and robustness. section 14.4.2
Timing detectors on abalone data
A timing experiment evaluates twelve PyOD detectors across six dataset sizes. At 104,425 rows, prediction takes about 2,111.9 seconds for KDE, 318.6 for OCSVM, 12.1 for KNN, 12.0 for ABOD, and near-zero time for several simpler detectors. The measurements show why deployment workloads must be tested directly.
Listing 8.11 Sample of code to time fit and predict times
import time
from pyod.models.iforest import IForest
from pyod.models.lof import LOF
from pyod.models.ocsvm import OCSVM
from pyod.models.gmm import GMM
from pyod.models.kde import KDE
from pyod.models.knn import KNN
from pyod.models.hbos import HBOS
from pyod.models.ecod import ECOD
from pyod.models.copod import COPOD
from pyod.models.abod import ABOD
from pyod.models.cblof import CBLOF
from pyod.models.pca import PCA
def time_detector(clf):
start_time = time.process_time()
clf.fit(test_df)
iteration_fit_results_arr.append(time.process_time() - start_time)
start_time = time.process_time()
clf.decision_function(test_df)
iteration_predict_results_arr.append(time.process_time() - start_time)
data = fetch_openml('abalone', version=1) #1
df = pd.DataFrame(data.data, columns=data.feature_names)
df = pd.get_dummies(df)
df = pd.DataFrame(RobustScaler().fit_transform(df), columns=df.columns)
fit_results_arr = [] #2
predict_results_arr = []
for multiplier in [1, 5, 10, 15, 20, 25]: #3
test_df = pd.concat([df]*multiplier) #4
iteration_fit_results_arr = [len(test_df)]
iteration_predict_results_arr = [len(test_df)]
time_detector(clf = IForest()) #5
time_detector(clf = LOF())
time_detector(clf = OCSVM())
time_detector(clf = GMM())
time_detector(clf = KDE())
time_detector(clf = KNN())
time_detector(clf = HBOS())
time_detector(clf = ECOD())
time_detector(clf = COPOD())
time_detector(clf = ABOD())
time_detector(clf = CBLOF())
time_detector(clf = PCA())
fit_results_arr.append(iteration_fit_results_arr)
predict_results_arr.append(iteration_predict_results_arr)
col_names = ['Number Rows', 'IF', 'LOF', 'OCSVM',
'GMM', 'KDE', 'KNN', 'HBOS', 'ECOD',
'COPOD', 'ABOD', 'CBLOF', 'PCA']
fit_results_df = \
pd.DataFrame(fit_results_arr, columns=col_names)
display(fit_results_df)
predict_results_df = pd.DataFrame(predict_results_arr, columns=col_names)
display(predict_results_df)
Use the timing pattern to reproduce fit and prediction measurements across detectors and duplicated workload sizes. section 8.7
