liveBook · Manning

Proxy metric

Also called proxy signal

A proxy metric is an observable, measurable quantity or variable used to approximate a target attribute, quality, or outcome that is difficult to measure directly.

How 8 Manning books explain it

Machine Learning System Design

A measurable quantity that serves as an indirect approximation of a desired outcome that is difficult to measure directly.

How it is used
Uses proxy metrics to assess progress toward hard-to-measure business outcomes before deployment and during experiments.
How deep it goes
Explains how to decompose goals, map metric hierarchies, and assess proxy relationships through offline and online evaluation.
Examples
Illustrates proxies with recommender conversion and retail average check as a conditional stand-in for revenue.
What you can do after
Evaluate and select proxy metrics by linking them to goals, making assumptions explicit, and comparing offline indicators with online outcomes.

Read how Machine Learning System Design explains proxy metric →

Hidden Influences

A measurable quantity used as an indirect representation of a less directly observable target construct.

How it is used
Uses proxy metrics to connect measurable platform behavior with hard-to-observe user value and organizational goals.
How deep it goes
Explains construct validity and optimization incentives through examples, without specifying a procedure for selecting metrics.
Examples
Uses inflammatory posts, click-through rates, purchase behavior, wine prices, and active-user goals to illustrate indirect measurement.
What you can do after
Evaluate proxy metrics by considering whether their definitions and incentives align with the intended goal.

Read how Hidden Influences explains proxy metric →

Data-Driven Cybersecurity

A metric drawn from other business measurements that provides cybersecurity insight when considered with direct or derived metrics.

How it is used
Adds business context to direct or derived cybersecurity metrics for risk and resource decisions.
How deep it goes
Defines indirect metrics and illustrates their interpretation, without specifying a calculation method common to all proxies.
Examples
Uses training-related incident reduction, insurance premiums, third-party audit scores, system uptime, and resource usage.
What you can do after
Evaluate/understand a proxy alongside direct evidence and decide whether its insight warrants collection or investigation.

Read how Data-Driven Cybersecurity explains proxy metric →

AI Model Evaluation

A proxy metric is an indirect measure whose value is intended to indicate a target quality or outcome that is difficult to measure directly.

How it is used
Connects model behavior to product outcomes through measurable stand-ins that can guide iteration before lagging outcomes mature.
How deep it goes
Defines metric hierarchy layers and explains proxy failure, validation, and revision as models and products evolve.
Examples
Uses movie recommendation clicks, watch-time, retention, genre diversity, and cohort fairness to illustrate a metric hierarchy.
What you can do after
Evaluate and revise proxies against outcomes, diagnostics, and historical correlations rather than treating a single engagement metric as success.

Read how AI Model Evaluation explains proxy metric →

Reinforcement Learning from Human Feedback

A proxy metric is an imperfect measurable quantity used to approximate a desired quality or objective.

How it is used
Uses measurable reward signals to optimize model behavior when human satisfaction cannot be specified directly.
How deep it goes
Explains proxy divergence, Goodhart-style over-optimization, symptoms, measurement relationships, and practical mitigations.
Examples
Uses verbose confident responses, rare-token repetition, sycophancy, hedging, repetitiveness, and over-refusal as failure examples.
What you can do after
Diagnose over-optimization by comparing proxy scores with downstream evaluations and human judgments, then apply regularization or data improvements.

Read how Reinforcement Learning from Human Feedback explains proxy metric →

Causal AI

A proxy variable is an observed variable that indirectly encodes information about another attribute of interest.

How it is used
Uses observed variables to encode latent confounders or expose indirect influence from protected attributes in predictive systems.
How deep it goes
Defines indirect encoding, then works through neural proxy construction and counterfactual analysis; stops short of a complete fairness solution.
Examples
Uses latent vector Z, names, neighborhoods, social networks, shopping habits, and loan classifications as proxy examples.
What you can do after
Evaluate models by tracing indirect attribute encoding and testing whether predictions change under counterfactual protected-attribute interventions.

Read how Causal AI explains proxy metric →

Machines that Think

A proxy signal is a measurable indicator used to approximate a broader objective that cannot be directly measured.

How it is used
Uses measurable signals to approximate objectives and examines proxy features that can carry historical bias into predictions.
How deep it goes
Explains how proxy optimization and inherited data patterns can diverge from intended goals, with illustrative cases rather than a formal measurement procedure.
Examples
Illustrates approval-driven agreement and healthcare spending used to estimate medical need.
What you can do after
Evaluate and understand how a measurable signal can diverge from its intended objective or encode unequal historical outcomes.

Read how Machines that Think explains proxy metric →

Vibe Engineering

A proxy metric is an observable measure that indirectly indicates a value or outcome that cannot be measured directly.

How it is used
Replaces developer self-reports with observable indicators when evaluating AI-assisted coding value.
How deep it goes
Defines correlation without causation and compares three operational metrics, while warning against single-metric interpretation.
Examples
Uses cycle-time reduction, pull-request growth, shrinking pull-request size, and first-attempt review success.
What you can do after
Evaluate AI coding impact through observable metrics while checking confounders and avoiding causal overclaims.

Read how Vibe Engineering explains proxy metric →

Also covered in

What to know first

Related concepts

Questions about proxy metric

What is proxy metric?

A proxy metric is an observable, measurable quantity or variable used to approximate a target attribute, quality, or outcome that is difficult to measure directly.

Which Manning books explain proxy metric?

Machine Learning System Design, Hidden Influences, Data-Driven Cybersecurity, AI Model Evaluation, Reinforcement Learning from Human Feedback, Causal AI, Machines that Think and Vibe Engineering.

What should I know before learning proxy metric?

Manning books that teach proxy metric usually introduce language model, deep learning, cross entropy loss and hallucination first.

What kinds of proxy metric are there?

Kinds of proxy metric include engagement rate.

What is proxy metric a kind of?

Proxy metric is a kind of metric. A metric is a quantitative measure that assigns values to a property or outcome so it can be assessed, compared, or monitored.

What is proxy metric part of?

Proxy metric is part of metric hierarchy. A metric hierarchy is an organized structure that decomposes high-level objectives into progressively more specific metrics and establishes their relative priorities.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.