Data integrity preserves the relationships between data elements so the dataset remains coherent and meaningful. Before analyzing text, ask whether each review belongs to the right product and order, represents the intended period, and appears only once. section 2.2 section 5.2.1
Protect integrity by checking the dataset systematically:
- Remove duplicate reviews, including repeated or slightly altered copies that could distort results. section 5.2.1
- Decide how to handle missing values through imputation, exclusion, or removal. section 5.2.1
- Check that records are relevant, current, consistently formatted, and written in an expected language. section 5.2.1 section 2.2
A misleading review count
A system accidentally copies one customer’s complaint every second for a full minute. Treating those copies as separate opinions gives that complaint disproportionate weight. Similarly, reviews scraped from a retailer may concern a competitor, or may not be tied to any order. section 5.2.1
