Descriptive statistics reduce a data set to a few comparable numerical attributes. A summary statistic is one number that describes a set of numbers, such as a mean or median, while accepting that some information is lost. section 6.1.1 section 6.1.2
Use descriptive statistics as an initial EDA work item:
- Generate standardized summaries, such as counts, means, medians, and other numeric attributes, to establish a first view of the data. section 6.1.1 section 6.1.2
- Choose data-frame processing for relatively small data sets, or use SQL when moving data into memory is inconvenient or the scale is impractical. section 6.1.2
- Treat summaries as evidence rather than a complete account: compare them with plots because identical statistics can conceal different distributions or relationships. section 6.1.3
From measurements to decisions
The values {1,2,3,4,5} have a mean and median of 3. Similarly, a pallet of flour may contain varied exact weights, while its mean weight of 1.5 KG provides enough information for a buyer estimating its price. section 6.1.2
select count(*), year from temperature_readings group_by year >(52560000, 52560000, 52560000, 525704000, 52560000)
Use grouped SQL counts to inspect whether records are distributed as expected across a time field before modeling. section 6.1.2
