An importance score is a numeric value that ranks how significant or urgent a flagged outlier is. It complements a category explaining why the record was flagged and reflects project relevance rather than statistical unusualness alone. section 15.1 section 15.3.1
Rule-based scoring can proceed by:
- Assigning explanations when measurable conditions apply, such as unusually long sessions or high error rates. section 15.3.1
- Adding a numeric weight for each applicable category, either by counting known anomalies or using category-specific importance values. section 15.3.1
- Testing the rules on flagged records and borderline or highly scored records before using them to sort future investigations. section 15.3.1
Listing 15.6 Sorting outliers detected in KDD Cup dataset
top_scored_df = df[df['IF Scores'] > 0.1].copy()
top_scored_df['Total Errors'] = top_scored_df[error_cols].sum(axis=1)
top_scored_df['Label'] = 'Unknown'
top_scored_df['Importance'] = 0
for i in top_scored_df.index:
row = top_scored_df.loc[i]
expl = ''
importance = 0
if row['duration'] > 1500:
expl += "Unusually long sessions. "
importance += 1
if row['Total Errors'] > 200:
expl += "Unusually high error rates. "
importance += 2
if len(expl):
top_scored_df.loc[i, 'Label'] = expl
top_scored_df.loc[i, 'Importance'] = importance
The listing demonstrates assigning explanations and accumulating importance for detected records using duration and total-error thresholds. section 15.3.1
