liveBook · Manning

Metric hierarchy

A metric hierarchy is an organized structure that decomposes high-level objectives into progressively more specific metrics and establishes their relative priorities.

How 2 Manning books explain it

Machine Learning System Design

An organized structure that decomposes a high-level objective into progressively more specific measurable metrics.

How it is used
Uses metric hierarchies to guide and justify metric selection during ML system design.
How deep it goes
Illustrates relationships and suggests goal decomposition, while placing creation of a company-wide hierarchy outside ML system design.
Examples
Traces retail clicks through conversions and purchases to GMV, and contrasts recommendation engagement with profitability.
What you can do after
Evaluate metric choices against a decomposed goal and identify potential proxies and conflicting outcomes.

Read how Machine Learning System Design explains metric hierarchy →

AI Model Evaluation

A metric hierarchy is an ordered structure of metrics that establishes their relative priority when evaluating trade-offs.

How it is used
Aligns model evaluation with product strategy when metrics move in conflicting directions.
How deep it goes
Defines three metric layers and their roles, then stops short of prescribing a universal measurement formula.
Examples
Uses movie recommendation retention, CTR, watch time, genre diversity, and cohort fairness as concrete measures.
What you can do after
Evaluate model changes by separating durable outcomes, fast proxies, and diagnostics that explain regressions or hidden harms.

Read how AI Model Evaluation explains metric hierarchy →

Related concepts

Questions about metric hierarchy

What is metric hierarchy?

A metric hierarchy is an organized structure that decomposes high-level objectives into progressively more specific metrics and establishes their relative priorities.

Which Manning books explain metric hierarchy?

Machine Learning System Design and AI Model Evaluation.

What is metric hierarchy part of?

Metric hierarchy is part of metric framework. A metric framework is a structured set of evaluation metrics and rules that connects model objectives with product goals and constraints.

What are the parts of metric hierarchy?

Metric hierarchy includes diagnostic metric, proxy metric, outcome metric and performance.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.