A rule is an explicit test applied to a data row. When its conditions hold, it can flag the row, add to a score, or assign a label. Rules are useful when the pattern of interest can be specified in advance; they do not require the pattern to be statistically unusual. section 4.3.2 section 4.2.2
Choose and run rules according to the task:
- For detection, test known conditions directly. Rules can run sequentially, stopping at the first match, or in parallel, allowing several matches for one row. section 4.3.2
- For sorting, apply rules after a detector has flagged records. Assign categories that help investigators group and prioritize results; rules are often preferred to classifiers when both work equally well because they are easier to interpret. section 15.3 section 15.3.3
- Combine rules with detectors when unknown patterns matter. Rules are fast, deterministic, and interpretable, but a fixed set cannot anticipate every new form of fraud or other anomaly. section 4.3.2 section 1.1.1
Expense checks
An expense rule might immediately flag a transaction outside permitted account types or hours. Other tests can add points for frequent recent transactions or a large meals charge, flagging the row only when the total exceeds a limit. These conditions express what warrants review, not proof that fraud occurred. section 4.3.2 section 1.1.1
score = 0
if Account not in (Meals, Travel, Supplies, Payroll, Inventory, Rent)
return True
if Time before 7:00am or after 8:00pm
return True
if Count in Prev. Hour > 10:
score += 4
if Count in Prev. Hour <= 10 and > 5:
score += 3
if Amount > 200.00
score += 2
if Account is Meals and Amount > 100.00
score += 5
if Account is Meals and Amount > 80.00
score += 4
if score > 6:
return True
return False
The expense-rule listing contrasts immediate returns with weighted tests and a final score threshold. section 4.3.2
Set thresholds by examining the data, then tune them against the results. For flagged network sessions, summed error-rate features and session duration can suggest labels such as high error rates or unusually long sessions. A label need not explain everything: if a record scores much higher than others in its category, investigate whether it has an additional, unknown anomaly or an extreme instance of the known one. section 15.3.1 section 15.3.3
Listing 15.6 Sorting outliers detected in KDD Cup dataset
top_scored_df = df[df['IF Scores'] > 0.1].copy()
top_scored_df['Total Errors'] = top_scored_df[error_cols].sum(axis=1)
top_scored_df['Label'] = 'Unknown'
top_scored_df['Importance'] = 0
for i in top_scored_df.index:
row = top_scored_df.loc[i]
expl = ''
importance = 0
if row['duration'] > 1500:
expl += "Unusually long sessions. "
importance += 1
if row['Total Errors'] > 200:
expl += "Unusually high error rates. "
importance += 2
if len(expl):
top_scored_df.loc[i, 'Label'] = expl
top_scored_df.loc[i, 'Importance'] = importance
The sorting example applies duration and total-error tests to detector-flagged rows, retaining a label and importance value. section 15.3.1
