A fraud alert has a cost even when no fraud occurred: it can disrupt a customer and consume an investigator’s time. Precision asks how trustworthy an alert is. It is the proportion of transactions flagged as fraud that actually are fraud. section 7.4.1 section 8.1.1
For a rule or model, precision = fraudulent transactions flagged ÷ all transactions flagged. A high-precision rule is usually right when it fires, but it may still miss most fraud. Compare precision with recall, which measures the share of actual fraud caught. F1 score combines the two; area under the precision-recall curve evaluates performance across decision thresholds. section 7.4.1 section 8.1.1
Choosing a fraud rule
Test a rule such as “high-value transaction from a brand-new account” against labeled transactions. Count the fraudulent and legitimate transactions it flags, then divide the fraudulent count by the total flagged count. A broader rule may catch more fraud yet produce enough false positives to overwhelm investigators or frustrate customers. Precision alone cannot settle that choice. section 7.4.1
Listing 7.2 Applying and evaluating simple fraud detection rules
def calculate_rule_detection_performance(df, rules_conditions):
"""
Applies a dictionary of rules (as pandas query strings) to a DataFrame
and calculates performance metrics for each rule.
"""
results_list = []
for rule_name, condition_query_str in rules_conditions.items():
# Filter the DataFrame based on the rule's condition
flagged_by_rule_df = df.query(condition_query_str)
num_total_flagged = len(flagged_by_rule_df)
if num_total_flagged > 0:
num_fraud_flagged = flagged_by_rule_df['label'].sum() # Sum of 1s (fraud)
num_legit_flagged = num_total_flagged - num_fraud_flagged
# Detection Rate (Precision): Proportion of flagged transactions that are fraud
detection_rate = num_fraud_flagged / num_total_flagged
else:
num_fraud_flagged = 0
num_legit_flagged = 0
detection_rate = 0.0
results_list.append({
'Rule Name': rule_name,
'Legitimate Transactions Flagged': num_legit_flagged,
'Fraudulent Transactions Flagged': num_fraud_flagged,
'Precision (Fraud/Flagged)': detection_rate
})
return pd.DataFrame(results_list)
# Define our set of expert rules using pandas query syntax
defined_rules = {
'High Value Transaction (>5000)': "transaction_amount > 5000",
'Multiple Recent Payment Methods (>=3)': "recent_7d_payment_methods_count >= 3",
'Frequent Login Failures Recently (>=3)': "num_recent_login_failures >= 3",
'New Account & Large Transaction (>3000, age<=30d)': "(account_age_days <= 30) and (transaction_amount > 3000)",
'International IP & High Value Transaction (>2000)': "(is_domestic_ip == False) and (transaction_amount > 2000)"
}
# Evaluate the rules on our sample data
rule_performance_df = calculate_rule_detection_performance(sample_data.copy(), defined_rules)
print("\nPerformance of Defined Fraud Detection Rules:")
display(rule_performance_df)
The Python listing applies rule conditions to a pandas DataFrame and calculates each rule’s detection rate, its name for precision, from the flagged transactions. section 7.4.1
Precision matters especially when fraud is rare. An always-approve model can appear accurate while catching no fraud; conversely, an aggressive flagging policy can burden legitimate customers. Review precision alongside missed fraud, investigation capacity, and the business costs of each error before promoting a rule to production. section 7.4.1 section 8.1.1
