Entity resolution determines which records represent the same customer or other real-world entity when data is spread across sources, identifiers are incomplete, or names differ. It turns a collection of records into a reusable entity-level data model for analysis such as customer counting. section 3.0 section 3.4.2
Start by inspecting completeness, duplicates, and the meaning of each source. Standardize comparable fields such as names and postcodes, merge records with shared IDs where available, and preserve source flags so the resulting model shows where each record came from. section 3.4.2
A pragmatic deduplication workflow is:
- Remove fully identical rows, then assign placeholder IDs to guest records that lack customer IDs. section 3.4.3
- Define an exact-match rule using available attributes, such as first name, surname, and postcode; group matching records and mark one as the main record. section 3.4.3
- Try approximate linkage as a later iteration: block candidate pairs by postcode, compare names with Damerau-Levenshtein similarity, and retain pairs meeting every chosen comparison. section 3.4.3
- Compare the results of exact and approximate methods, investigate disagreements, and state the assumptions behind the count. section 3.4.3 section 3.4.4
Customer records across sales, CRM, and customer databases
A company may store “South West Motors” in one database and “South West Motors Limited” in another, while other customers appear only as guest checkout details or as IDs without names. Exact name-and-postcode matching yields 27,394 main records from 35,395 rows, but assumes equal names and postcodes identify one customer and only exact duplicates exist. Fuzzy matching can surface likely typo-level matches for review. section 3.0 section 3.4.3
Figure 3.36 Instances where the two deduplication methods disagree
Inspect records where exact duplicate grouping and approximate record linkage disagree before treating either method as authoritative. section 3.4.3
Resolution is a best-guess decision rather than perfect automation. Incomplete records may still be suitable for counting customers but unsuitable for segmentation; edge cases and uncertainty should remain visible to stakeholders and guide the next iteration. section 3.4.4
