liveBook · Manning

Missing value

As explained in The Well-Grounded Data Analyst

A missing value is an absent or unavailable value for a field that is expected to contain data.

A missing value is an absent or unavailable field value. Treat it as a data-quality finding, not automatically as an error: its importance depends on the field’s role in the analysis and on why it is absent. section 2.2.2 section 6.1.2

Start by counting missing values by column, then inspect affected rows and the data-generating context. Missing session or brand values may be harmless for a product-focused analysis, while missing addresses can block geographic extraction. section 4.4.1 section 6.4.1

Choose a response according to the analytical need and the evidence available:

  • Drop rows when a required value is unavailable and the loss is acceptably small; 968 missing addresses out of 100,000 rows were removed before city analysis. section 2.2.2
  • Keep rows when the missing field does not affect the current question, such as unknown sessions or brands in product exploration. section 4.4.1
  • Assign an explicit placeholder when preserving the record matters, such as filling a missing street with “-- NO STREET INFORMATION --”. section 10.4.1
  • Investigate high or unexpected missingness rather than treating it as random; survey omissions can reflect nonresponse, early exit, or conditional questions. section 6.1.2 section 6.4.1

Missing address data in a customer analysis

Before extracting cities from customer addresses, count nulls. With just under 1% missing addresses, dropping those rows is a workable compromise. At higher rates, investigate the cause, group records as “Other,” or seek another customer data source. section 2.2.2

Figure 6.7 Missing values for each column
figure

A per-column missing-value table helps identify which survey fields need contextual investigation before categorical analysis. section 6.4.1

How The Well-Grounded Data Analyst covers it

How deep it goes
Demonstrates counting, inspecting, dropping, retaining, and filling missing values, while deferring any universal threshold for acceptable loss.
Examples
Uses customer addresses, ecommerce events, survey responses, and property transactions with missing address fields.
What it assumes
Assumes ability to inspect tabular columns and rows while teaching missingness as a context-dependent analytical decision.
What you can do after
Evaluate/understand whether to retain, drop, label, investigate, or supplement records with absent values.
How it connects
Connects missing values to data quality, exploratory data analysis, categorical data, survey data, and data dictionaries.

Also explained in

Compare how 35 books explain missing value →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of The Well-Grounded Data Analyst. The section links point to the parts of the book it draws on.