liveBook · Manning

Missing value

As explained in The Quick Python Book, Fourth Edition

A missing value is a data item for which the intended observation is unavailable, inapplicable, or not reported.

A missing value marks an observation that is unavailable, inapplicable, or not reported. Raw data may encode it as the string "Missing"; replacing that marker with None, an empty string, or pandas NaN makes its status explicit. section 21.4.1 section 24.3.1

Handle missing values deliberately when cleaning data:

  • Convert "Missing" while reading the file, or guard every calculation so it skips that string. section 21.4.1
  • Treat missing heat-index fields as distinct from percentage fields: both may be parsed as strings, but percentages require numeric translation before averaging. section 21.4.1 section 21.6
  • Standardize varied markers such as NA, N/A, ?, and - by supplying them together as pandas read_csv() na_values. section 24.3.1

Converting a raw marker

Use `temp = pd.read_csv("temp_data_01.csv", na_values=['Missing'])`. A field containing `Missing` then appears as `NaN`, while ordinary temperature and count values remain numeric. The resulting missing value can be excluded or handled explicitly during later calculations. section 24.3.1

 [None, 'Illinois', 17, 'Jan 01, 1979', '1979/01/01', 17.48, 994, 6, 30.5,
2.89, 994, -13.6, 15.8, 'Missing', 0, 'Missing', 'Missing', '0.00%']

Inspect a representative weather row to see missing heat-index values beside numeric temperatures, counts, and a percentage marker. section 21.4.1

How The Quick Python Book, Fourth Edition covers it

How deep it goes
Explains raw markers, replacement choices, parsing effects, and pandas standardization, but stops short of implementing averaging logic.
Examples
Uses Illinois weather records, heat-index columns, the Missing marker, pandas NaN, and varied no-data spellings.
What you can do after
Apply it via pandas read_csv() to convert selected textual markers into NaN before inspecting or calculating with columns.
How it connects
Connects missing values with data cleaning, pandas, CSV files, numeric parsing, and percentage conversion.

Also explained in

Compare how 35 books explain missing value →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of The Quick Python Book, Fourth Edition. The section links point to the parts of the book it draws on.