A fraud team wants to catch suspicious transactions without sending legitimate customers into unnecessary reviews. The false-positive rate measures the share of actual negative cases incorrectly classified as positive: false positives ÷ all actual negatives. For fraud detection, that means legitimate transactions flagged ÷ all legitimate transactions. A lower rate reduces customer friction and wasted investigation effort, but rules still need to catch fraud. section 7.4.1
Check the denominator before comparing rule metrics. A rule evaluation can count fraudulent and legitimate transactions among those flagged and report precision—the share of flagged transactions that are fraud. That is not the false-positive rate, whose denominator includes all legitimate transactions, whether flagged or not. Review both alongside the amount of fraud caught: a narrow rule may produce accurate alerts yet miss most fraud, while a broad rule may catch more fraud and overwhelm investigators. section 7.4.1
Listing 7.2 Applying and evaluating simple fraud detection rules
def calculate_rule_detection_performance(df, rules_conditions):
"""
Applies a dictionary of rules (as pandas query strings) to a DataFrame
and calculates performance metrics for each rule.
"""
results_list = []
for rule_name, condition_query_str in rules_conditions.items():
# Filter the DataFrame based on the rule's condition
flagged_by_rule_df = df.query(condition_query_str)
num_total_flagged = len(flagged_by_rule_df)
if num_total_flagged > 0:
num_fraud_flagged = flagged_by_rule_df['label'].sum() # Sum of 1s (fraud)
num_legit_flagged = num_total_flagged - num_fraud_flagged
# Detection Rate (Precision): Proportion of flagged transactions that are fraud
detection_rate = num_fraud_flagged / num_total_flagged
else:
num_fraud_flagged = 0
num_legit_flagged = 0
detection_rate = 0.0
results_list.append({
'Rule Name': rule_name,
'Legitimate Transactions Flagged': num_legit_flagged,
'Fraudulent Transactions Flagged': num_fraud_flagged,
'Precision (Fraud/Flagged)': detection_rate
})
return pd.DataFrame(results_list)
# Define our set of expert rules using pandas query syntax
defined_rules = {
'High Value Transaction (>5000)': "transaction_amount > 5000",
'Multiple Recent Payment Methods (>=3)': "recent_7d_payment_methods_count >= 3",
'Frequent Login Failures Recently (>=3)': "num_recent_login_failures >= 3",
'New Account & Large Transaction (>3000, age<=30d)': "(account_age_days <= 30) and (transaction_amount > 3000)",
'International IP & High Value Transaction (>2000)': "(is_domestic_ip == False) and (transaction_amount > 2000)"
}
# Evaluate the rules on our sample data
rule_performance_df = calculate_rule_detection_performance(sample_data.copy(), defined_rules)
print("\nPerformance of Defined Fraud Detection Rules:")
display(rule_performance_df)
The rule-evaluation listing counts legitimate transactions flagged by each rule and calculates precision. Compare its flagged counts with the full set of legitimate transactions when assessing false-positive rate. section 7.4.1
Treat the rate as an operational measure, not an isolated model score. In anti-money-laundering work, success is assessed alongside case resolution time and avoided compliance penalties; one reported system cut false AML positives by 60%. That reduction describes the number of false alerts, not necessarily a 60% reduction in false-positive rate. In production fraud defenses, capture investigation outcomes to refine rules and thresholds, and monitor customer impact, investigation cost, drift, fairness, and governance over time. section 2.4.2 section 7.4.4
