A percentile marks a value’s position in an ordered set: the pth percentile indicates the point at or below which a specified proportion of observations falls. Percentiles can summarize detector scores or define robust outlier boundaries. section 14.6.3 section 2.1.2
Use the percentile choice to control how scores or observations influence the result:
- A high percentile, such as the 90th, approximates the maximum while reducing dependence on one detector; with 100 detectors, it uses the tenth-highest score. section 14.6.3
- A low percentile, such as the 25th, rewards agreement among many detectors, but very low choices can amplify arbitrary scores from detectors that found nothing anomalous. section 14.6.3
- The interquartile range uses the 25th and 75th percentiles: IQR = Q3 − Q1, with outlier limits commonly set at Q1 − 2.2IQR and Q3 + 2.2IQR. section 2.1.2
Comparing ensemble summaries
For detector scores 15.1, 0.8, 0.3, and 1.1, the 25th percentile is 0.8 and the 75th percentile is 1.1, while the mean of the top two is 8.1. These summaries emphasize consensus and extreme detector support differently. section 14.6.3
scores_df['perc_90'] = scores_df[scores_cols].quantile(0.90, axis=1)
Compute a row-wise 90th percentile across detector columns with pandas quantile. section 14.6.3
import pandas as pd import numpy as np data = pd.Series(np.random.normal(size=10_000)) q1 = data.quantile(0.25) q3 = data.quantile(0.75) iqr = q3 - q1 iqr_lower_limit = q1 - (2.2 * iqr) iqr_upper_limit = q3 + (2.2 * iqr)
Calculate Q1, Q3, IQR, and percentile-based outlier limits with pandas and NumPy. section 2.1.2
