Data segmentation divides records into smaller groups intended to contain comparable items. For outlier detection, a segment gives each record a more relevant definition of normal: a payroll transaction need not be judged against sales transactions. Segments can come from known categories or bins of a numeric feature rather than a clustering algorithm. section 3.2.3
A practical segmentation workflow is:
- Choose a field that reflects a meaningful difference in how records arise, such as transaction type or expense account. section 3.2.3 section 10.2.3
- Model or assess each subset separately, so behavior in one group does not redefine normal behavior in another. section 3.2.3 section 10.2.3
- Check whether the groups remain large enough for sound comparisons and whether a full-dataset assessment would preserve useful signals shared across groups. section 3.2.3
Check a fund’s assigned segment
Calculate a fund’s outlier score relative to its assigned comparison group, then calculate its score against other groups. A lower score elsewhere suggests the assignment deserves review under the chosen similarity measure; it does not establish a universally correct classification. An unsuitable comparison group can make performance evaluations misleading. section 1.1.10
Table 10.7 Staff expenses (view table figure)
| Row |
Staff ID |
Department |
Account |
Date of expense |
Date submitted |
Time submitted |
Amount |
|---|---|---|---|---|---|---|---|
| 1 |
9000483 |
Sales |
Meals |
02/03/2023 |
02/03/2023 |
09:10:21 |
12.44 |
| 2 |
9303332 |
Marketing |
Travel |
02/03/2023 |
02/03/2023 |
10:43:35 |
41.90 |
| 3 |
9847421 |
Engineering |
Meals |
02/03/2023 |
02/03/2023 |
10:56:04 |
643.99 |
| 4 |
9303332 |
Marketing |
Supplies |
02/03/2023 |
02/03/2023 |
11:12:09 |
212.00 |
The staff-expenses rows include an Account field with meals, travel, and supplies. Splitting on that field illustrates how separate, simpler expense models could replace one model covering every account type. section 10.2.3
Manual segments can be quicker to establish and easier to interpret than clusters, but splitting creates more models to test. Smaller reference datasets may speed prediction for methods such as KNN and LOF, and separate models may ease memory pressure and tuning; total training time need not fall. Repeated subdivision can leave too few records for reliable comparison. section 3.2.3 section 10.2.3
