Pareto frontier

As explained in Delivering Machine Learning Projects: From design to deployment

A Pareto front is the boundary formed by nondominated outcomes that represent the best available tradeoffs among multiple criteria.

A Pareto frontier is the boundary of candidate models with the best available tradeoffs across multiple tests. It helps combine independent measures without forcing them into one aggregate ranking. A model is excluded when another is at least as good across the compared dimensions and better in one of them. section 8.3.3

Comparing hbi and rotm performance

For imaginary models evaluated on the hard-but-important (hbi) and run-of-the-mill (rotm) test sets, model F remains in the Pareto set because it offers a best tradeoff. Model G is excluded because it is no better than F on either test. The resulting candidate set narrows the decision but does not determine the final choice. section 8.3.3

Figure 8.3 A Pareto front for models evaluated from the hard-but-important (hbi) and run-of-the-mill (rotm) data sets

Use the figure to inspect the Pareto-front boundary for models evaluated across hbi and rotm. section 8.3.3

Pareto sets can handle more than two tests, but they usually leave several candidates for qualitative judgment. Record the decision basis when selecting among remaining models, especially when quantitative results do not identify a clear winner. section 8.6

How Delivering Machine Learning Projects: From design to deployment covers it

How deep it goes

Defines Pareto-efficient sets, demonstrates dominance with two tests, and extends the method to many tests before final judgment.

Examples

Uses imaginary models evaluated on hbi and rotm, including F's inclusion and G's exclusion from the Pareto set.

What you can do after

Evaluate/understand model candidates, narrow the selection pool, and document the judgment among remaining tradeoffs.

How it connects

Connects model selection with rankings, multi-criteria decision making, cross-validation, A/B testing, and multi-armed bandits.

liveBookmanning.com

Read Delivering Machine Learning Projects: From design to deployment on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Bayesian Optimization in ActionA Pareto frontier is the boundary formed by solutions that are not dominated by any other feasible solution across all objectives.
• 

  liveBookmanning.com
  Building LLM Applications with DSPyA Pareto front is a set of candidate solutions that are not uniformly outperformed across the objectives being evaluated.

Compare how 3 books explain Pareto frontier →

Related concepts

• GEPAused by · GEPA (Genetic-Pareto) is a reflective evolutionary optimization method that generates and evaluates candidate prompts using task information and feedback.
• hillclimbingcontrasts with · Hill climbing is an iterative local-search technique that moves from a candidate solution to a neighboring solution with a better objective value.
• Pareto optimizationproduced by · A multi-objective optimization technique that seeks multiple nondominated trade-off solutions rather than one scalar objective optimum.
• nondominated solutionhas part · A feasible multiobjective solution for which no other feasible solution is at least as good in every objective and strictly better in one.
• validation setuses · A validation set is held-out data used to assess a model during development and guide model selection or tuning, rather than to fit its parameters.

This explainer was generated by Manning's knowledge graph from the text of Delivering Machine Learning Projects: From design to deployment. The section links point to the parts of the book it draws on.