liveBook · Manning

Personal data

As explained in Effective Data Analysis

Personal data, also called personal information or personally identifiable information, is information that identifies or can reasonably be linked to an individual.

Personal data is information that can identify an individual. Names, addresses, phone numbers, email addresses, and IP addresses are common examples. In analytics, even routine website-visit records may contain identifying details, so a useful dataset can also pose a privacy risk. section 8.2.1 section 8.1.1

Personally identifiable information (PII) is one category of sensitive data. Protected health information (PHI) concerns health information that can be tied to an individual. The distinction matters when choosing safeguards, but both call for care when disclosure could harm someone. section 8.3 section 8.4

Start with the purpose of the analysis, then reduce the personal data it requires:

  • Collect only necessary fields, keep them only as long as needed, restrict raw-data access, and avoid sending sensitive extracts through insecure channels. section 8.3.1
  • Remove identifying fields that the analysis does not need. Where broader geography answers the question, retain standardized city, state, or zip code rather than a full address. section 8.3.2
  • When records must be linked across sources, consider pseudonymization: use a coded identifier for analysis and keep identifying information separately with restricted access. This differs from anonymization, which removes identifying information rather than retaining a link. section 8.3.2 section 8.4

Preparing transaction data

A transaction table might contain a customer’s name, phone number, street address, and email. An analyst can drop the name and phone, extract useful geographic fields before dropping the address, and replace the email in the analysis table with a hash. The email-to-hash table stays separate and secure. Treat the resulting table cautiously: coded or apparently anonymized records can still be re-associated with individuals. section 8.3.2 section 8.4

Table 8.1 Examples of anonymized and pseudonymized data
Example
Anonymized
Pseudonymized
Medical research
Patients have names, birthdays, and contact information removed from the data warehouse.
Patients have names replaced with pseudonyms and a unique identifier that can be tied to their contact information.
Human resources management
Employees have names, social security numbers, and birthdays masked from view.
Employees have all identifying information stored in a separate table with restricted access.
Customer surveys
Respondents have no identifying information collected.
Respondents can share their emails for future studies, which are only accessible by survey tool admins.

Compare anonymized and pseudonymized records in medical research, human resources, and customer surveys. section 8.3.2

Handling personal data also involves obligations beyond transforming a table. Applicable regulations address storage, use, deletion, access, and individual rights; research participants should be told what information is collected and how it will be protected. Analysts may need to coordinate practices with security, IT, and legal teams. section 8.2.1 section 8.1.2

How Effective Data Analysis covers it

How deep it goes
Defines identifying data and works through field removal, geographic reduction, and pseudonymization without prescribing one compliance procedure for every jurisdiction.
Examples
Uses website metadata and a transaction table containing names, phone numbers, addresses, and emails.
What it assumes
Assumes an analytics context while defining sensitive-data categories and illustrating transformations on a transaction dataset.
What you can do after
Apply it via pandas to remove unnecessary identifiers, retain less granular location data, and separate hashed email identifiers from originals.
How it connects
Connects personally identifiable information with protected health information, data minimization, informed consent, anonymization, pseudonymization, and data protection regulations.

Also explained in

Compare how 9 books explain personal data →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Effective Data Analysis. The section links point to the parts of the book it draws on.