Data quality
Data quality is the degree to which data meets requirements for accuracy, completeness, consistency, and suitability for its intended use.
Starting Data Analytics with Generative AI and Python
A data quality issue is a defect in data's accuracy, completeness, consistency, validity, or interpretation.
- How it is used
- Establishes whether available data can support reliable business analysis and decisions.
- How deep it goes
- Develops a structured assessment using inspection, cleaning, completeness checks, and clarification, while recognizing that quality cannot be fully automated.
- Examples
- Examines customer identifier relationships and discusses biased reviews, anonymization, and domain-specific stopwords.
- What you can do after
- Evaluate and understand data quality by inspecting variables, checking relationships, and seeking clarification before drawing conclusions.
Read how Starting Data Analytics with Generative AI and Python explains data quality →
Effective Data Analysis
Data quality is the degree to which data is accurate, complete, consistent, timely, and fit for its intended analytical purpose.
- How it is used
- Uses data quality as a foundation for reliable analysis and organizational decision-making.
- How deep it goes
- Defines four quality dimensions and explains checks, missing-data responses, and proactive pipeline testing.
- Examples
- Uses page-view identifiers, time-zone consistency, month-end financial data, and forecast-data tests to illustrate quality requirements.
- What you can do after
- Apply data quality checks using Python and pytest to detect duplicate dates, invalid ranges, and missing forecast values.
LLM Customization and Fine-Tuning
The degree to which adaptation data is correct, relevant, and representative enough to support reliable model behavior.
- How it is used
- Guides enterprise data readiness and iterative curation for model adaptation.
- How deep it goes
- Provides a curation checklist and acquisition tradeoffs, including privacy, distribution, annotation agreement, and synthetic-data quality gates.
- Examples
- Uses a curated IT support dataset and a healthcare case with 8% of examples removed during filtering.
- What you can do after
- Evaluate and improve adaptation datasets using review, distribution checks, privacy safeguards, and held-out evaluation.
Read how LLM Customization and Fine-Tuning explains data quality →
Machine Learning System Design
Data quality is the degree to which data is accurate, complete, valid, consistent, timely, and suitable for its intended purpose.
- How it is used
- Treats data quality as a pipeline and production-monitoring concern for reliable ML predictions.
- How deep it goes
- Gives preprocessing, monitoring, alerting, and fallback actions, including a specific missing-data baseline and thresholds.
- Examples
- Uses a retail demand-forecasting pipeline with transaction duplicates, gaps in competitor prices, and feature-level missing-data alerts.
- What you can do after
- Diagnose model degradation by checking data quality separately from genuine changes in the patterns being modeled.
Read how Machine Learning System Design explains data quality →
Data Without Labels
Data quality is the degree to which data is complete, valid, accurate, representative, available, consistent, timely, and internally coherent for a particular use case.
- How it is used
- Positions data quality as a gate between identifying a business problem and preparing data for modeling.
- How deep it goes
- Defines core quality attributes and gives task-specific checks, but defers detailed quality inspection to a later preparation step.
- Examples
- Uses daily rainfall measurements, annual sales completeness, date-range validity, and stakeholder-verified sales and customer indicators.
- What you can do after
- Evaluate/understand whether a dataset fits a use case and which quality checks must precede preparation and modeling.
Architecting for Autonomy
Data quality is the degree to which information is accurate, complete, timely, consistent, traceable, private, confidential, and compliant for a specific use.
- How it is used
- Sets the practical limit on delegated authority by testing whether information is fit to influence an agent’s decisions and actions.
- How deep it goes
- Defines agentic quality dimensions and gives verification checks for sources, transformations, retrieval, and decision episodes.
- Examples
- Uses fabricated legal authorities in Mata v. Avianca and checks on legal records, transcripts, and supporting authorities.
- What you can do after
- Evaluate whether information is fit to influence an agent for a given purpose and level of autonomy.
The Art of AI Product Development
Data quality is the degree to which data is accurate, complete, consistent, timely, and fit for its intended purpose.
- How it is used
- Supports training-data preparation and language-model output reliability for AI products.
- How deep it goes
- Explains annotation guidelines, agreement checks, class balance, bias, noise, privacy, and knowledge-cutoff risks.
- Examples
- Uses fashion recommendations, labeled classifier examples, biased job associations, and content-generation fact checking.
- What you can do after
- Diagnose training-data and generated-content risks, then apply labeling checks, human review, validation, and fresher data sources.
Read how The Art of AI Product Development explains data quality →
Snowflake Data Engineering
Data quality is the degree to which data is accurate, complete, consistent, valid, and fit for its intended purpose.
- How it is used
- Uses validation, testing, logging, and alerting to keep scheduled data pipelines reliable and fit for downstream consumption.
- How deep it goes
- Explains synchronous and asynchronous tests, violation responses, metric functions, event logging, and alerts without detailing individual quality-rule implementations.
- Examples
- Demonstrates primary-key validation, logging-table test results, scheduled metric functions, event tables, alerts, and stored-procedure pipeline loads.
- What you can do after
- Apply it via Snowflake tasks, data metric functions, logging tables, and alerts to detect violations and trigger operational responses.
Delivering Machine Learning Projects: From design to deployment
Data quality is the degree to which data is accurate, complete, consistent, relevant, timely, and fit for its intended use.
- How it is used
- Uses data quality to locate risks during an early ML data survey before modeling and application delivery.
- How deep it goes
- Inspects records, missingness, placeholders, and visible patterns, while deferring deeper investigation to exploratory data analysis.
- Examples
- Uses smart-building incident text, SQL record samples, missing-value checks, and image collections containing potentially irrelevant content.
- What you can do after
- Evaluate whether data contains actionable information and decide what to investigate, resolve, or avoid.
Read how Delivering Machine Learning Projects: From design to deployment explains data quality →
Timeless Algorithms, The Foundational Ideas
Data quality is the degree to which data are accurate, relevant, consistent, and sufficiently complete for their intended use.
- How it is used
- Frames training-data suitability as a constraint on learning and deployment, distinct from dataset size.
- How deep it goes
- Distinguishes accuracy, consistency, relevance, coverage, and informational value without defining a quantitative quality metric.
- Examples
- Uses daytime-only pedestrian images, duplicated internet documents, and errors in collected text to illustrate data limitations.
- What you can do after
- Evaluate training collections for inaccurate labels, inconsistent measurements, missing coverage, duplication, and deployment mismatch.
Read how Timeless Algorithms, The Foundational Ideas explains data quality →
Healthcare IT
Data quality is the degree to which data is accurate, complete, consistent, timely, and fit for its intended clinical or operational purpose.
- How it is used
- Frames precise, trustworthy data as a design responsibility for healthcare products, workflows, and interoperable systems.
- How deep it goes
- Connects quality to workflow fit, trust, and patient impact but stops short of defining measurement procedures or validation metrics.
- Examples
- Uses NHS National Programme for IT, Google Health, and Microsoft HealthVault to illustrate quality and trust failures.
- What you can do after
- Evaluate healthcare data designs against precision, workflow fit, trust, and patient impact.
Architecting an Apache Iceberg Lakehouse
Data quality is the degree to which data is accurate, consistent, complete, timely, and fit for its intended use.
- How it is used
- Guides stakeholder discovery, ingestion design, and consumption-layer architecture by connecting data quality with trust, freshness, and usability.
- How deep it goes
- Connects quality to architectural tradeoffs and shared consumption, but stops short of defining validation rules or quality metrics.
- Examples
- Uses reports, models, dashboards, BI tools, notebooks, ML platforms, and replicated departmental datasets as examples.
- What you can do after
- Evaluate architecture choices by relating ingestion and consumption patterns to data trust, consistency, latency, and duplication.
Read how Architecting an Apache Iceberg Lakehouse explains data quality →
Build AI Drug Discovery Pipelines
Data quality is the degree to which data is accurate, consistent, complete, reliable, and fit for its intended analytical purpose.
- How it is used
- Frames reliable data as a prerequisite for trustworthy AI-assisted drug discovery and model predictions.
- How deep it goes
- Explains measurement noise and representation bias with concrete cases, but does not provide a remediation workflow.
- Examples
- Uses cross-laboratory ADMET variation, protein-database composition, chemical-class imbalance, and demographic underrepresentation as examples.
- What you can do after
- Evaluate model reliability by distinguishing algorithmic limitations from measurement noise and training-data underrepresentation.
Read how Build AI Drug Discovery Pipelines explains data quality →
Machine Learning Platform Engineering
Data quality is the degree to which data is accurate, complete, consistent, timely, valid, and fit for its intended purpose.
- How it is used
- Runs quality assurance before model training to prevent defective structured data from entering production ML workflows.
- How deep it goes
- Defines schema and type checks with runnable validation, while deferring extensive exploration and deeper defect analysis.
- Examples
- Uses a MinIO-hosted Parquet dataset and checks for user_id, item_id, and rating columns before training.
- What you can do after
- Diagnose production input defects with schema and data-type checks before launching training runs.
Read how Machine Learning Platform Engineering explains data quality →
Financial AI in Practice
Data quality is the degree to which data is accurate, complete, consistent, timely, and fit for its intended purpose.
- How it is used
- Protects financial AI and risk decisions by ensuring inputs are accurate, timely, complete, and governed.
- How deep it goes
- Explains failure consequences and pipeline controls, but gives no explicit data-quality measurement procedure.
- Examples
- Uses noisy or outdated LLM inputs, credit data feeds, and real-time loan-application features as examples.
- What you can do after
- Operate a data asset layer that governs feeds, transformations, privacy constraints, and task-specific stores.
Also covered in
- Learn AI Data EngineeringIn early access (MEAP)
- Generative AI in Action
- Digital Twins in Action
- Graph Neural Networks in Action
- Grokking AI Algorithms, Second Edition
- Building Specialized AI SystemsIn early access (MEAP)
- First-Party Data Activation
- Outlier Detection in Python
- Architecture Modernization: Socio-technical alignment of software, strategy, and structure
- Data-Driven Cybersecurity
- Demand Forecasting Best Practices
- Coding with AI
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- complianceCompliance is conformity with applicable legal, regulatory, contractual, policy, or other governing requirements.
- machine learningMachine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
- missing valueA missing value is an expected data value or observation that is absent, unavailable, unknown, or inapplicable.
Related concepts
- accuracyhas part · Accuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- completenesshas part · The degree to which all elements required by a defined scope or objective are present, represented, or covered.
- validityhas part · Validity is the degree to which data, measures, outputs, or evaluations satisfy the requirements and accurately represent the intended use or construct.
- consistencycontrasts with · Consistency is the degree to which related data, behavior, or results agree with one another and conform to applicable rules or expectations.
- data representativenesshas part · Data representativeness is the degree to which a dataset reflects the relevant population, distribution, diversity, and conditions to which a model or conclusion is intended to apply.
Questions about data quality
What is data quality?
Data quality is the degree to which data meets requirements for accuracy, completeness, consistency, and suitability for its intended use.
Which Manning books explain data quality?
Starting Data Analytics with Generative AI and Python, Effective Data Analysis, LLM Customization and Fine-Tuning, Machine Learning System Design, Data Without Labels, Architecting for Autonomy, The Art of AI Product Development, Snowflake Data Engineering and 7 more.
What should I know before learning data quality?
Manning books that teach data quality usually introduce language model, compliance, machine learning and missing value first.
What is the difference between data quality and data volume?
Data quality is the degree to which data meets requirements for accuracy, completeness, consistency, and suitability for its intended use. Data volume is the quantity of data produced, available, stored, transferred, or processed within a defined scope or interval.
What are the parts of data quality?
Data quality includes accuracy, completeness, validity and data representativeness.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.














