A threshold is a cutoff that decides which observations qualify for an analysis. It turns a broad question—such as which products truly improved, which cycling locations merit attention, or which property sales belong in a buyer-facing app—into an explicit inclusion rule. The choice is a practical judgment: it should fit the decision being supported, and changing it can change the result set. section 4.4.3 section 9.2.1 section 10.4.1
Start by inspecting the distribution and context of the measure rather than choosing a number blindly. Histograms and descriptive statistics reveal skew, sparsity, tails, and plausible breakpoints. Quartiles can show what values represent the upper half or upper quarter of observations, but they do not determine whether that level is useful; the required level depends on the business question and resulting recommendations. section 9.2.1 section 10.4.1
Use thresholds as an iterative analytical decision:
- Define the population to retain or exclude and name the rule clearly, such as a minimum volume, minimum percentage, or maximum price. section 4.4.3 section 10.4.1
- Check unusual records before filtering them away, so that apparent outliers can be understood as special cases, data issues, or valid but out-of-scope observations. section 10.4.1 section 4.4.3
- Combine thresholds when one measure alone is misleading; percentage growth, for example, may need minimum baseline and current volumes. section 4.4.3
- Record assumptions and inspect how many observations remain, because an arbitrary cutoff changes the findings and may need revision. section 4.4.3 section 9.2.1
Filtering meaningful product growth
A product that sells only a handful of units can show a large percentage increase that is not commercially meaningful. One rule retains products with more than 10 November sales and more than 200% December growth, while separately retaining products with no November sales only when December sales exceed 100. These conditions yielded 449 products for category, subcategory, and brand analysis. The values are not universal: they deliberately trade a broader list for a list of products that more clearly outperformed themselves. section 4.4.3
Selecting cycling locations with several criteria
A cycling-location analysis first examines total traffic and its quartiles to choose a plausible traffic cutoff. It then identifies locations where cycling makes up at least 10% of traffic, locations with at least 50% growth, and locations whose peak hour is 8 or 17. Intersecting those sets produces a narrower list for recommendations. Thresholds therefore work not only as exclusions, but as components of a transparent eligibility definition. section 9.2.1
Keeping residentially relevant sales
Property transactions above £10 million are removed from an app aimed at residential buyers after inspection shows that the extreme values include entire developments rather than individual properties. Lower-value category B transactions remain because they may still matter to users interested in repossessions or other nonstandard local sales. A threshold can thus exclude data for audience fit without treating every unusual value as invalid. section 10.4.1
DEC_VS_NOV_PCT_CUTOFF = 200
NOV_VOLUME_CUTOFF = 10
ONLY_DEC_VOLUME_CUTOFF = 100
december_high_performers = (
pd.concat(
[
league_table[(np.isinf(league_table["volume_diff_pct"]) == False)
& (league_table["november_volume"] > NOV_VOLUME_CUTOFF)
& (league_table["volume_diff_pct"] > DEC_VS_NOV_PCT_CUTOFF)],
league_table[(np.isinf(league_table["volume_diff_pct"]))
& (league_table["december_volume"] > ONLY_DEC_VOLUME_CUTOFF)]
],
axis=0,
ignore_index=True)
.merge(product_catalog.drop(columns="product_name"), on="product_id")
)
print(december_high_performers.shape)
This filter shows a threshold implemented as two branches: established products must clear minimum volume and growth rules, while zero-baseline products require a higher December volume. section 4.4.3
