liveBook · Manning

Standard deviation

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

Standard deviation measures the typical dispersion of values around their arithmetic mean.

Standard deviation describes how widely numeric values vary around their mean. In a dataset, it provides a quick sense of whether a feature is nearly constant or meaningfully dispersed. section 2.4.2

Using dispersion to screen features

For the Auto MPG numeric features, standard deviations include 7.82 for mpg, 105.21 for displacement, 38.52 for horsepower, 849.83 for weight, and 2.82 for acceleration. A feature with zero or very low variance may be excluded because it adds little information, provided missingness is considered separately. section 2.4.2

data[numeric_feats].std()

Compute dispersion for selected numeric columns with data[numeric_feats].std(). section 2.4.2

Standard deviation also supports comparison after standardization: subtract each feature’s mean and divide by its standard deviation before inspecting boxplots. section 2.4.2

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

How deep it goes
Defines dispersion operationally and applies pandas computation, stopping short of mathematical derivation.
Examples
Uses Auto MPG features including mpg, displacement, horsepower, weight, and acceleration.
What you can do after
Apply it via pandas to screen low-variance features and prepare comparable standardized columns.
How it connects
Connects standard deviation with feature selection, missing values, standardization, and boxplots.

Also explained in

Compare how 15 books explain standard deviation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.