In this chapter
In statistics, we’re often interested in structured data, typically tabular data, for use with analysis tools such as pandas and Excel. Unstructured data, such as documents, audio, or images, where each pixel acts as a variable (or three variables, if you count the RGB values), is often impractical for manual statistical analysis because of the sheer number of variables involved. Such data is best left to perceptual models, such as deep learning and large language models. Statistics play a significant role in these models, but in a way that’s largely inaccessible to internal analysis. Still, we can verify the outputs with statistics.
The examples in this chapter often focus on a single dataset for a single variable, such as temperatures or exam scores. However, when working with real-world data, you will usually encounter tabular data with multiple variables as columns, with each column representing a different attribute or measurement. You will see an example of this at the end of the chapter.
Before we get into computing measures with data, let’s start with some critical qualitative questions, such as “What is data anyway?”