Metric
A metric is a quantitative measure that assigns values to a property or outcome so it can be assessed, compared, or monitored.
Data-Driven Cybersecurity
A quantitative measure derived or interpreted using statistical methods to assess performance, conditions, or change under uncertainty.
- How it is used
- Connects security performance and risk measurements to business goals, resource priorities, and decisions.
- How deep it goes
- Defines selection criteria and develops a goal-to-measurement-to-improvement cycle, emphasizing interpretation rather than statistical formulas.
- Examples
- Uses patching time, vulnerability remediation, incident detection time, and industry-specific security measures to illustrate actionable choices.
- What you can do after
- Practice/perform selecting, benchmarking, reporting, and revisiting security metrics to guide improvements and communicate decisions.
Effective Data Analysis
A model evaluation metric is a quantitative measure of how well a statistical model fits data or performs its intended task.
- How it is used
- Uses evaluation metrics to assess model fit and predictive accuracy alongside coefficients and residuals.
- How deep it goes
- Defines several regression metrics and their diagnostic roles, while directing readers to further resources for differences between model classes.
- Examples
- Illustrates diagnostics with a housing-price regression using square footage and bedroom count.
- What you can do after
- Diagnose regression model fit using evaluation metrics and residuals, and compare model iterations using relative metrics.
Experimentation for Engineers
A quantitative measure of business performance, such as revenue, engagement, or time spent using an application.
- How it is used
- Guides acceptance or rejection of production changes through online measurements of business outcomes.
- How deep it goes
- Defines business metrics and develops selection, stakeholder, update, and multi-metric trade-off practices without deriving experimental statistics.
- Examples
- Uses daily revenue, DAU, swipe prediction, video comments, time spent, advertising revenue, and trading profit and risk.
- What you can do after
- Operate/configure metric-driven experiments by selecting business-specific measures, thresholds, and trade-offs for production decisions.
Logs and Telemetry
A metric is a usually numeric measurement with a descriptive label that represents the state or activity of hardware or software.
- How it is used
- Supports observability by representing resource and business measurements for near-real-time monitoring, aggregation, routing, and incident investigation.
- How deep it goes
- Explains collection, tagging, log conversion, and name-based filtering, while deferring detailed Prometheus metric-type documentation to its reference.
- Examples
- Demonstrates CPU, disk, memory, and network collection; dummy JSON logs converted to Prometheus metrics; and node_cpu metric selection.
- What you can do after
- Operate/configure it by collecting host measurements, deriving Prometheus metrics from logs, and restricting exported metric names.
AI Model Evaluation
A quantitative measure of model performance derived from behavior or outcomes observed during live operation.
- How it is used
- Guides live model and product decisions by measuring technical performance and user behavior in online experiments.
- How deep it goes
- Explains metric selection, offline alignment, monitoring formulas, alert thresholds, and interpretation pitfalls with a worked recommendation experiment.
- Examples
- Uses a movie recommendation latency test and a metric mirror table with click-rate, diversity, latency, and fairness watchpoints.
- What you can do after
- Evaluate and understand live model impact by selecting product-aligned metrics, defining watchpoints, and interpreting proxy limitations.
Evaluating AI Systems
A quantitative or scored measure used to assess a system against an evaluation criterion.
- How it is used
- Turns AI evaluation goals and dimensions into evidence for decisions about system behavior.
- How deep it goes
- Defines metric components, contrasts measurement categories, and outlines quality criteria and custom metric design.
- Examples
- Uses disease classification precision, retrieval-augmented generation faithfulness, and medical chatbot measurement trade-offs.
- What you can do after
- Evaluate and select complementary metrics for a decision, and outline a custom metric when existing measures miss requirements.
Automating API Delivery
A business metric is a quantitative measure of an organization's commercial performance, customer outcomes, quality, or cost.
- How it is used
- Guides API product improvement decisions by relating developer activity and experience to a business focus metric.
- How deep it goes
- Explains metric selection through PDSA, developer-journey touchpoints, and product categories; treats lower-level measures as hypothesized leading indicators.
- Examples
- Uses active-subscriber goals, landing-page visitors, referrals, sign-ups, daily active users, retention, and conversion rate.
- What you can do after
- Apply it via developer journey mapping and product analytics to choose, capture, share, and review API improvement measures.
Machine Learning System Design
An evaluation measure that can be calculated only after a system is deployed and its output is used in a real business context.
- How it is used
- Measures system effects in live business settings and relates them to final goals.
- How deep it goes
- Explains the distinction from offline proxies and outlines testing, correlation checks, and reporting considerations.
- Examples
- Uses recommender sales, search click-through and conversion rates, and spam complaints as concrete measures.
- What you can do after
- Evaluate system impact by selecting online measures, testing alternatives, and reporting observed business effects.
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A quantitative measure used to monitor or assess model performance rather than directly define the training objective.
- How it is used
- Monitors model training and guides feature selection, hyperparameter optimization, and early stopping.
- How deep it goes
- Distinguishes metrics from optimization losses and outlines their operational uses, but does not define particular metric formulas.
- Examples
- Uses XGBoost training, feature selection, hyperparameter search spaces, cross-validation schemes, and early stopping as practical contexts.
- What you can do after
- Evaluate and tune models by comparing configurations, monitoring training, testing generalization, and stopping unproductive iterations.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains metric →
Building LLM Applications with DSPy
A criterion or measurement used to assess the quality of a language-model application's outputs.
- How it is used
- Guides prompt and model comparison, baseline assessment, and production-readiness decisions.
- How deep it goes
- Defines custom response scoring and explains the trade-off between a single optimization metric and multiple final-evaluation metrics.
- Examples
- Uses airline intent classification to contrast exact correctness with partial credit for similar labels or spelling differences.
- What you can do after
- Apply metrics via DSPy to score responses and evaluate a final program across multiple dimensions.
Read how Building LLM Applications with DSPy explains metric →
ScyllaDB in Action
A metric is a numeric measurement of a system, database, or workload property collected for observation over time.
- How it is used
- Supports monitoring a ScyllaDB cluster by collecting observable measurements for dashboards, alerts, load testing, and incident response.
- How deep it goes
- Defines metrics through concrete node measurements, time-series collection, PromQL queries, and dashboard interpretation without detailing metric instrumentation.
- Examples
- Uses cache-partition insertions, 95th-percentile read latency, reads in progress, compactions, and Grafana CQL-read graphs.
- What you can do after
- Diagnose/debug with metrics by querying time-series data and relating latency changes to reads or compactions.
Build and Orchestrator in Go (From Scratch)
A metric is a measured value describing the state or performance of a worker machine.
- How it is used
- Supports task placement by reporting each worker’s current resource state to a manager.
- How deep it goes
- Builds a periodic collector and JSON endpoint, while deferring a deep explanation of the Linux /proc filesystem.
- Examples
- Uses a restaurant seating analogy, Linux /proc file output, a CollectStats loop, and a GET /stats route.
- What you can do after
- Apply it via goprocinfo and a GET /stats endpoint to expose a worker’s current resource state.
Read how Build and Orchestrator in Go (From Scratch) explains metric →
Think Like a CTO
A metric is a quantifiable measure used to assess a condition, activity, or outcome.
- How it is used
- Guides team reporting, accountability, hiring decisions, and operational monitoring with measures suited to each purpose.
- How deep it goes
- Explains selection, collection, review cadence, and comparison, but gives no formulas or fixed numerical thresholds.
- Examples
- Uses support resolution times and interaction counts, ticket and release measures, and CPU, memory, and disk usage.
- What you can do after
- Practice/perform it by choosing team-specific measures, reviewing them regularly, comparing appropriate baselines, and investigating deviations.
Vibe Engineering
A metric is a quantitative measure used to assess a property or outcome of a software system or process.
- How it is used
- Uses quantitative evidence to validate generated outcomes, locate performance bottlenecks, and monitor production health.
- How deep it goes
- Explains aggregate metrics and timer instrumentation down to percentiles, invocation counts, and measurement granularity, while cautioning against indiscriminate coverage.
- Examples
- Demonstrates query-accuracy comparisons, loadFile and scan timers, p99 latency, error rates, input-size distributions, and production latency spikes.
- What you can do after
- Diagnose bottlenecks and evaluate changes using benchmarks, targeted timers, accuracy comparisons, and production aggregates.
Machine Learning Platform Engineering
A custom metric is an application-defined quantitative measurement exposed for monitoring a specific behavior or outcome.
- How it is used
- Tracks application behavior that default service metrics miss, supporting operational alerting and reporting.
- How deep it goes
- Defines metric types, records observations in service code, verifies /metrics exposure, and visualizes results in Grafana.
- Examples
- Uses object-detection confidence histograms and recommendation-request counters for concrete monitoring cases.
- What you can do after
- Build it from scratch with prometheus-client, expose observations from inference code, and query them in Grafana.
Read how Machine Learning Platform Engineering explains metric →
Think Like a Software Engineering Manager
A metric is a quantitative measure used to assess performance against defined success criteria.
- How it is used
- Uses quantitative measures to assess engineering-manager and cross-functional team performance against defined project or organizational goals.
- How deep it goes
- Defines metrics as objective gauges and lists performance dimensions, but does not provide a general method for selecting or calculating them.
- Examples
- Illustrates success with a product launch targeting 20% higher customer satisfaction and 30% fewer escalations, plus platform reliability and cost measures.
- What you can do after
- Evaluate engineering-management and cross-functional outcomes against business, delivery, quality, collaboration, and customer measures.
Read how Think Like a Software Engineering Manager explains metric →
Build High Value Software Systems
A metric is a quantitative measure of a property or outcome of a process, system, or activity.
- How it is used
- Makes hidden queues and process stalls visible so teams can investigate system-level causes.
- How deep it goes
- Distinguishes learning signals from rankings and targets through software-delivery examples, without detailing measurement calculations.
- Examples
- Uses cycle-time spikes caused by review delays and build-time targets that remove integration tests.
- What you can do after
- Evaluate whether a measure supports learning or encourages behavior that worsens delivery.
Read how Build High Value Software Systems explains metric →
gRPC Microservices in Go
A metric is a numerical measurement of a system or application property collected over time.
- How it is used
- Tracks microservice behavior over time and supports dashboards and alerts that expose operational problems.
- How deep it goes
- Defines Prometheus metric fields and grouping, then examines quality indicators and the limitations of average latency.
- Examples
- Uses a service memory-usage alert and contrasting request latencies to show alerting and reporting pitfalls.
- What you can do after
- Apply it via OpenTelemetry instrumentation and Prometheus collection to monitor services and configure metric-based alerts.
Deep Learning with R, Third Edition
A custom metric is a task-specific quantitative measure defined to evaluate success according to particular project goals.
- How it is used
- Defines project success measures and monitors model performance during training and evaluation.
- How deep it goes
- Explains custom Keras metrics through subclassing, state updates, result reporting, and resetting.
- Examples
- Contrasts classification measures and demonstrates a sparse-label RMSE metric.
- What you can do after
- Build a custom Keras metric by subclassing Metric and implementing state updates, result reporting, and reset behavior.
Read how Deep Learning with R, Third Edition explains metric →
Effective Conversational AI
A metric is a quantitative measure used to evaluate a system's performance or outcomes.
- How it is used
- Guides iterative conversational-system improvement by tracking outcomes tied to business goals and user satisfaction.
- How deep it goes
- Connects metric selection to monitoring and improvement decisions through deployment comparisons and a business-impact example.
- Examples
- Uses deployment failure comparisons and an insurance claim flow measured by opt-outs, escalations, task completion, and user scores.
- What you can do after
- Evaluate whether fixes worked by comparing outcomes across deployments and use relevant measures to target improvements.
Also covered in
- The Well-Grounded Data Analyst
- Demand Forecasting Best Practices
- Designing Deep Learning Systems: A guide for software engineers
- Microservices Patterns, Second EditionIn early access (MEAP)
- Interview Speak
- Classic Game Programming on the NES: Make your own retro video game
- Applied Reinforcement LearningIn early access (MEAP)
- Context EngineeringIn early access (MEAP)
- Software Engineering for Data ScientistsIn early access (MEAP)
- Designing AI SystemsIn early access (MEAP)
- LLM Evaluation and Alignment, The Foundational IdeasIn early access (MEAP)
- Rearchitecting LLMsIn early access (MEAP)
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- meanThe mean is a measure of central location equal to the arithmetic average of numerical observations, or the probability-weighted average of a random variable’s possible values.
- performancePerformance is a measure of how effectively a person, process, model, or system achieves its objectives, including qualities such as output quality, speed, and resource efficiency.
- customer satisfactionCustomer satisfaction is the degree to which customers or users evaluate a product, service, interaction, or outcome as meeting their needs and expectations.
Related concepts
- evaluationused by · Evaluation is the systematic assessment of a system, model, process, or outcome against relevant criteria or objectives using evidence.
- optimizationused by · Optimization is the systematic selection or modification of values, configurations, or implementations to improve a defined objective, often subject to constraints.
- evaluation criterioncontrasts with · A defined condition, standard, or measure used to assess an outcome, option, system, or decision.
- mean absolute percentage errorincludes · Mean absolute percentage error is the average absolute prediction error divided by the corresponding actual value, expressed as a percentage.
- agreement metricincludes · An agreement metric quantitatively measures consistency between judgments, predictions, or outputs and other judgments, outputs, or a reference.
Questions about metric
What is metric?
A metric is a quantitative measure that assigns values to a property or outcome so it can be assessed, compared, or monitored.
Which Manning books explain metric?
Data-Driven Cybersecurity, Effective Data Analysis, Experimentation for Engineers, Logs and Telemetry, AI Model Evaluation, Evaluating AI Systems, Automating API Delivery, Machine Learning System Design and 12 more.
What should I know before learning metric?
Manning books that teach metric usually introduce language model, mean, performance and customer satisfaction first.
What is the difference between metric and evaluation criterion?
A metric is a quantitative measure that assigns values to a property or outcome so it can be assessed, compared, or monitored. A defined condition, standard, or measure used to assess an outcome, option, system, or decision.
What kinds of metric are there?
Kinds of metric include mean absolute percentage error, activity metric and agreement metric.
What is metric also called?
Metric is also called score.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



















