liveBook · Manning

Data integrity

As explained in Statistics Slam Dunk

Data integrity is the accuracy, consistency, completeness, and validity of data throughout its collection and processing.

Data integrity means keeping data accurate, consistent, complete, and valid while preparing it for analysis. In a basketball salary dataset, integrity checks begin by retaining only needed variables, converting categories to factors, and inspecting summaries for blanks and uneven coverage. section 12.4

A practical integrity workflow includes:

  • Select salary, season_end, and team instead of carrying unnecessary variables. section 12.4
  • Convert season_end and team to factors, then use summary() to inspect distributions and group counts. section 12.4
  • Investigate four blank team values and recognize that early seasons have unusually few salary records. section 12.4
  • Exclude seasons from 1985 through 1990 before ranking salaries within each season and team. section 12.4

Checking suspicious records

A SQL query finds four records with blank team values, all from 1985. Comparing Boston Celtics records from 1987 and Los Angeles Lakers records from 1990 confirms that some early team-season groups contain only three or four observations, limiting reliable comparison. section 12.4

How Statistics Slam Dunk covers it

How deep it goes
Works through selection, factor conversion, summaries, anomaly queries, coverage filtering, and grouped ranking.
Examples
Uses blank team records, sparse early team-season groups, and salary rankings for Denver Nuggets and Chicago Bulls.
What it assumes
Assumes basic R and data-frame familiarity while demonstrating dplyr, SQL filtering, factors, summaries, and grouped ranking.
What you can do after
Diagnose data-quality problems and prepare grouped records for a defensible comparison.
How it connects
Connects data cleaning with descriptive statistics, missing data, SQL, dplyr, grouping, and ranking.

Also explained in

Compare how 18 books explain data integrity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Statistics Slam Dunk. The section links point to the parts of the book it draws on.