A normalized score expresses an observed category count relative to a reference count, producing a unitless measure of rarity. One option divides the actual count by the count expected under a uniform distribution; others divide by the mode or median count. section 2.2
Use normalized counts to make categorical rarity easier to compare:
- Compute unique values and their counts, then compare each observed count with its reference count. section 2.2
- Apply a threshold to label unusually rare values; the unitless scale permits a threshold that is not tied to the current dataset. section 2.2
- Treat the score as a screening measure rather than proof of an outlier, since simple rarity methods can fail when many values are uncommon. section 2.2
Cumulative rarity threshold
For the SpeedDating age_o feature, count each value, order counts from least to most frequent, and calculate cumulative fractions. A 0.5% cutoff identifies values whose cumulative count falls below that threshold.
