Proxy metric
A proxy metric is an observable, measurable quantity or variable used to approximate a target attribute, quality, or outcome that is difficult to measure directly.
Machine Learning System Design
A measurable quantity that serves as an indirect approximation of a desired outcome that is difficult to measure directly.
- How it is used
- Provides measurable stand-ins for business outcomes before deployment and intermediate measures within a hierarchy of goals.
- How deep it goes
- Explains selection through goal decomposition and offline-to-online prediction, then illustrates an experimentally checked proxy without prescribing a universal formula.
- Examples
- Uses an e-commerce recommender’s sales goal, offline-versus-online metric pairs, and average check as a conditional revenue proxy.
- What you can do after
- Evaluate/understand it by choosing an early measure, stating its link to the goal, and checking that link in an online experiment.
Read how Machine Learning System Design explains proxy metric →
Data-Driven Cybersecurity
A metric drawn from other business measurements that provides cybersecurity insight when considered with direct or derived metrics.
- How it is used
- Adds business context to direct or derived cybersecurity metrics for risk and resource decisions.
- How deep it goes
- Defines indirect metrics and illustrates their interpretation, without specifying a calculation method common to all proxies.
- Examples
- Uses training-related incident reduction, insurance premiums, third-party audit scores, system uptime, and resource usage.
- What you can do after
- Evaluate/understand a proxy alongside direct evidence and decide whether its insight warrants collection or investigation.
Reinforcement Learning from Human Feedback
A proxy metric is an imperfect measurable quantity used to approximate a desired quality or objective.
- How it is used
- Uses measurable reward signals to optimize model behavior when human satisfaction cannot be specified directly.
- How deep it goes
- Explains proxy divergence, Goodhart-style over-optimization, symptoms, measurement relationships, and practical mitigations.
- Examples
- Uses verbose confident responses, rare-token repetition, sycophancy, hedging, repetitiveness, and over-refusal as failure examples.
- What you can do after
- Diagnose over-optimization by comparing proxy scores with downstream evaluations and human judgments, then apply regularization or data improvements.
Read how Reinforcement Learning from Human Feedback explains proxy metric →
Causal AI
A proxy variable is an observed variable that indirectly encodes information about another attribute of interest.
- How it is used
- Uses observed variables to encode latent confounders or expose indirect influence from protected attributes in predictive systems.
- How deep it goes
- Defines indirect encoding, then works through neural proxy construction and counterfactual analysis; stops short of a complete fairness solution.
- Examples
- Uses latent vector Z, names, neighborhoods, social networks, shopping habits, and loan classifications as proxy examples.
- What you can do after
- Evaluate models by tracing indirect attribute encoding and testing whether predictions change under counterfactual protected-attribute interventions.
Related concepts
- metric hierarchypart of · A metric hierarchy is an organized structure that decomposes high-level objectives into progressively more specific metrics and establishes their relative priorities.
- reward hackingused by · Reward hacking is behavior that exploits weaknesses or mismatches in a reward or evaluation specification to obtain a high measured score without achieving the intended objective.
- click-through rateuses · Click-through rate is the proportion of displayed or offered items that receive a click.
- offline metricincludes · An offline metric is a quantitative measure of model or algorithm behavior computed using fixed historical, validation, or test data rather than live interaction.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.