Data lineage preserves the traceability of where a record came from after data is combined. It prevents deduplication from erasing meaningful identifiers or hiding the assumptions used to treat several records as one customer. section 3.3.1
When combining customer data, retain lineage by:
- Keeping all encountered identifiers when otherwise matching records may represent the same customer. section 3.3.1
- Using a separate indicator for each source rather than one exclusive source column, because a record may occur in multiple datasets. section 3.4.2
- Recording business-relevant origins such as guest checkout status alongside source-presence flags. section 3.4.2
A merged customer record
Jane Smith may be customer 8834 in one dataset and 931 in another. Retaining both identifiers and flags for purchase, CRM, and customer-database presence lets users trace the merged entity back to its inputs and assess the matching decision. section 3.3.1 section 3.4.2
Table 3.5 The data model schema that all data sources need to be transformed into
|
Column
|
Description
|
|---|---|
customer_id
|
The unique ID of the customer record, or NULL for guests.
|
first_name
|
Either from the customer or CRM tables or the guest information.
|
surname
|
Either from the customer or CRM tables or the guest information.
|
postcode
|
Either from the customer or CRM tables or the guest information.
|
age
|
From the customer or CRM tables, unavailable for guests.
|
is_guest
|
True if the data comes from a guest checkout.
|
in_purchase_data
|
True if this customer record appears in the purchase table. It is not exclusive since the customer could also appear in the customer or CRM datasets.
|
in_crm_data
|
True if the customer record exists in the CRM database.
|
in_customer_data
|
True if the customer record exists in the customer database.
|
The common customer schema makes lineage explicit through guest status and non-exclusive source-presence columns. section 3.4.2
