Entropy-based outlier detection flags observations whose removal produces unusually large reductions in the combined entropy of feature values. It scores each cell value independently and combines the values into a row score, so it does not model associations between features. section 7.9
Entropy measures uncertainty in a distribution. A categorical feature with many equally frequent values has higher entropy than one dominated by a single value; removing an unusual row can therefore make the remaining distributions more concentrated. section 7.9
Apply the technique with these practical steps:
- Represent categorical values directly and bin numeric features before calculating entropy. section 7.9
- Calculate entropy from normalized value frequencies, using the negative sum of p log2(p). section 7.9
- Interpret detections cautiously because the method emphasizes univariate rarity rather than unusual feature combinations. section 7.9
Color-valued features
For 300 rows, a feature containing 100 red, 100 blue, and 100 green values has high entropy. A feature containing 290 red and 10 blue has low entropy, so a row contributing a rare value may stand out when its removal reduces total entropy. section 7.9
