liveBook · Manning

Damerau–Levenshtein distance

As explained in The Well-Grounded Data Analyst

Damerau–Levenshtein distance is an edit-distance measure that counts character insertions, deletions, substitutions, and transpositions needed to transform one string into another.

Damerau–Levenshtein distance measures how many character edits—insertions, deletions, substitutions, and transpositions—transform one string into another. Larger distances indicate less similarity. section 3.4.3

In customer deduplication, it supports fuzzy comparison when names may contain typing differences. Candidate pairs can first be narrowed by postcode to reduce comparisons and balance accuracy against performance. section 3.4.3

A practical matching workflow is:

  • Block records by postcode to create candidate links. section 3.4.3
  • Compare first names and surnames with Damerau–Levenshtein thresholds, while matching postcodes exactly. section 3.4.3
  • Keep pairs that satisfy all comparison criteria, then link their customer IDs back to the customer dataset. section 3.4.3

Thresholded customer matching

A recordlinkage comparison can use Damerau–Levenshtein for first_name and surname with threshold 0.85, alongside an exact postcode comparison. Matching pairs then become linked duplicate records.

How The Well-Grounded Data Analyst covers it

How deep it goes
Defines edit distance and applies thresholded comparisons, without deriving the algorithm’s internal computation.
Examples
Uses first names, surnames, and postcodes in a thresholded customer-record comparison.
What it assumes
Assumes comparable customer fields and candidate-pair indexing; teaches edit distance as a string-similarity measure.
What you can do after
Apply it via recordlinkage to compare candidate customer names and combine fuzzy results with exact postcode matching.
How it connects
Connects fuzzy string comparison with entity resolution, record linkage, deduplication, exact matching, and blocking.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of The Well-Grounded Data Analyst. The section links point to the parts of the book it draws on.