Proxy metric
A proxy metric is an observable, measurable quantity or variable used to approximate a target attribute, quality, or outcome that is difficult to measure directly.
Machine Learning System Design
A measurable quantity that serves as an indirect approximation of a desired outcome that is difficult to measure directly.
- How it is used
- Uses proxy metrics to assess progress toward hard-to-measure business outcomes before deployment and during experiments.
- How deep it goes
- Explains how to decompose goals, map metric hierarchies, and assess proxy relationships through offline and online evaluation.
- Examples
- Illustrates proxies with recommender conversion and retail average check as a conditional stand-in for revenue.
- What you can do after
- Evaluate and select proxy metrics by linking them to goals, making assumptions explicit, and comparing offline indicators with online outcomes.
Read how Machine Learning System Design explains proxy metric →
Hidden Influences
A measurable quantity used as an indirect representation of a less directly observable target construct.
- How it is used
- Uses proxy metrics to connect measurable platform behavior with hard-to-observe user value and organizational goals.
- How deep it goes
- Explains construct validity and optimization incentives through examples, without specifying a procedure for selecting metrics.
- Examples
- Uses inflammatory posts, click-through rates, purchase behavior, wine prices, and active-user goals to illustrate indirect measurement.
- What you can do after
- Evaluate proxy metrics by considering whether their definitions and incentives align with the intended goal.
Data-Driven Cybersecurity
A metric drawn from other business measurements that provides cybersecurity insight when considered with direct or derived metrics.
- How it is used
- Adds business context to direct or derived cybersecurity metrics for risk and resource decisions.
- How deep it goes
- Defines indirect metrics and illustrates their interpretation, without specifying a calculation method common to all proxies.
- Examples
- Uses training-related incident reduction, insurance premiums, third-party audit scores, system uptime, and resource usage.
- What you can do after
- Evaluate/understand a proxy alongside direct evidence and decide whether its insight warrants collection or investigation.
AI Model Evaluation
A proxy metric is an indirect measure whose value is intended to indicate a target quality or outcome that is difficult to measure directly.
- How it is used
- Connects model behavior to product outcomes through measurable stand-ins that can guide iteration before lagging outcomes mature.
- How deep it goes
- Defines metric hierarchy layers and explains proxy failure, validation, and revision as models and products evolve.
- Examples
- Uses movie recommendation clicks, watch-time, retention, genre diversity, and cohort fairness to illustrate a metric hierarchy.
- What you can do after
- Evaluate and revise proxies against outcomes, diagnostics, and historical correlations rather than treating a single engagement metric as success.
Reinforcement Learning from Human Feedback
A proxy metric is an imperfect measurable quantity used to approximate a desired quality or objective.
- How it is used
- Uses measurable reward signals to optimize model behavior when human satisfaction cannot be specified directly.
- How deep it goes
- Explains proxy divergence, Goodhart-style over-optimization, symptoms, measurement relationships, and practical mitigations.
- Examples
- Uses verbose confident responses, rare-token repetition, sycophancy, hedging, repetitiveness, and over-refusal as failure examples.
- What you can do after
- Diagnose over-optimization by comparing proxy scores with downstream evaluations and human judgments, then apply regularization or data improvements.
Read how Reinforcement Learning from Human Feedback explains proxy metric →
Causal AI
A proxy variable is an observed variable that indirectly encodes information about another attribute of interest.
- How it is used
- Uses observed variables to encode latent confounders or expose indirect influence from protected attributes in predictive systems.
- How deep it goes
- Defines indirect encoding, then works through neural proxy construction and counterfactual analysis; stops short of a complete fairness solution.
- Examples
- Uses latent vector Z, names, neighborhoods, social networks, shopping habits, and loan classifications as proxy examples.
- What you can do after
- Evaluate models by tracing indirect attribute encoding and testing whether predictions change under counterfactual protected-attribute interventions.
Machines that Think
A proxy signal is a measurable indicator used to approximate a broader objective that cannot be directly measured.
- How it is used
- Uses measurable signals to approximate objectives and examines proxy features that can carry historical bias into predictions.
- How deep it goes
- Explains how proxy optimization and inherited data patterns can diverge from intended goals, with illustrative cases rather than a formal measurement procedure.
- Examples
- Illustrates approval-driven agreement and healthcare spending used to estimate medical need.
- What you can do after
- Evaluate and understand how a measurable signal can diverge from its intended objective or encode unequal historical outcomes.
Vibe Engineering
A proxy metric is an observable measure that indirectly indicates a value or outcome that cannot be measured directly.
- How it is used
- Replaces developer self-reports with observable indicators when evaluating AI-assisted coding value.
- How deep it goes
- Defines correlation without causation and compares three operational metrics, while warning against single-metric interpretation.
- Examples
- Uses cycle-time reduction, pull-request growth, shrinking pull-request size, and first-attempt review success.
- What you can do after
- Evaluate AI coding impact through observable metrics while checking confounders and avoiding causal overclaims.
Also covered in
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- deep learningDeep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
- cross entropy lossCross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
- hallucinationA hallucination is model-generated content that is false, fabricated, or unsupported by relevant evidence but presented as plausible or factual.
Related concepts
- metric hierarchypart of · A metric hierarchy is an organized structure that decomposes high-level objectives into progressively more specific metrics and establishes their relative priorities.
- reward hackingused by · Reward hacking is behavior that exploits weaknesses or mismatches in a reward or evaluation specification to obtain a high measured score without achieving the intended objective.
- metrickind of · A metric is a quantitative measure that assigns values to a property or outcome so it can be assessed, compared, or monitored.
- click-through rateuses · Click-through rate is the proportion of impressions or opportunities to display an item that result in a click on that item.
Questions about proxy metric
What is proxy metric?
A proxy metric is an observable, measurable quantity or variable used to approximate a target attribute, quality, or outcome that is difficult to measure directly.
Which Manning books explain proxy metric?
Machine Learning System Design, Hidden Influences, Data-Driven Cybersecurity, AI Model Evaluation, Reinforcement Learning from Human Feedback, Causal AI, Machines that Think and Vibe Engineering.
What should I know before learning proxy metric?
Manning books that teach proxy metric usually introduce language model, deep learning, cross entropy loss and hallucination first.
What kinds of proxy metric are there?
Kinds of proxy metric include engagement rate.
What is proxy metric a kind of?
Proxy metric is a kind of metric. A metric is a quantitative measure that assigns values to a property or outcome so it can be assessed, compared, or monitored.
What is proxy metric part of?
Proxy metric is part of metric hierarchy. A metric hierarchy is an organized structure that decomposes high-level objectives into progressively more specific metrics and establishes their relative priorities.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.







