Correlation
Correlation is a statistical relationship describing how two or more variables or quantities vary together, including the strength and direction of their association, without by itself implying causation.
Pandas Workout: 200 exercises to make you a stronger data analyst
A statistical measure of the strength and direction of association between the number of wines submitted by a country and its mean score.
- How it is used
- Uses correlation to investigate associations in grouped, joined, and cleaned real-world datasets.
- How deep it goes
- Defines direction and strength, interprets values from –1 to 1, and applies `corr()` alongside scatter plots.
- Examples
- Uses temperatures, oil prices, ice-cream prices, miles driven, wine scores, tourism spending, tuition, and admission rates.
- What you can do after
- Apply it via pandas `corr()` to quantify numeric associations and inspect them with scatter plots.
Read how Pandas Workout: 200 exercises to make you a stronger data analyst explains correlation →
Effective Data Analysis
Correlation is a statistical association describing how two variables vary together, without by itself establishing causation.
- How it is used
- Uses correlation to test associations in hypotheses and identify patterns that may inform predictive modeling.
- How deep it goes
- Defines the standardized coefficient, compares coefficient types, and works through interpretation and visualization without deriving formulas.
- Examples
- Uses daily rat sightings and weather correlations, plus a profit-margin exercise involving scatterplots, transformations, and regression.
- What you can do after
- Apply it via pandas to inspect associations, choose suitable coefficients, and communicate results without claiming causation.
Statistics Slam Dunk
A correlation coefficient is a numeric measure of the direction and strength of the linear relationship between two variables, ranging from -1 to +1.
- How it is used
- Screens numeric NBA variables for associations and potential linear-regression predictors.
- How deep it goes
- Defines coefficient interpretation, computes matrices and tests, and visualizes scatterplots without deriving the statistic.
- Examples
- Uses draft statistics, salaries and win shares, team payrolls and wins, and points scored or allowed versus wins.
- What you can do after
- Apply it via base R cor() and ggplot2 to compute, visualize, and compare numeric-variable associations.
Causal Inference for Data Science
Correlation is a statistical association in which two variables vary together without by itself establishing that either causes the other.
- How it is used
- Distinguishes observed association from evidence needed for causal decisions and supports predictive counterfactuals.
- How deep it goes
- Develops common-cause intuition, synthetic simulations, graph-based path calculations, and conditioning effects in linear models.
- Examples
- Uses city health, arcade revenue and doctorates, simulated population data, and paired movie-interest time series.
- What you can do after
- Evaluate/understand why an observed association may be confounded and how graph paths determine correlation in normalized linear models.
Read how Causal Inference for Data Science explains correlation →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
Bivariate correlation is a statistical measure of association between two variables or features.
- How it is used
- Screens numeric tabular features for redundancy and collinearity before feature selection or modeling.
- How deep it goes
- Explains pairwise matrices, thresholds, and feature-removal decisions, while contrasting correlation with broader VIF analysis.
- Examples
- Demonstrates synthetic near-duplicate numeric features and a pandas correlation-matrix seaborn heatmap.
- What you can do after
- Apply it via pandas and seaborn to flag correlated numeric-feature pairs for investigation and possible removal.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains correlation →
Grokking Statistics
A correlation coefficient is a numerical measure summarizing the strength and direction of association between two variables.
- How it is used
- Uses correlation to investigate relationships between paired variables before prediction with linear regression.
- How deep it goes
- Defines directional and linear association, interprets Pearson r, and demonstrates calculation, while deferring formula proofs.
- Examples
- Uses sleep and happiness, social media and happiness, rainfall, temperature and sports-drink sales, and spurious pool-drowning data.
- What you can do after
- Evaluate/understand Pearson correlation values, inspect linear patterns, and resist causal conclusions from association alone.
Build a Robo-Advisor with Python (From Scratch)
Correlation is a statistical measure of the direction and strength of dependence between two variables or asset returns.
- How it is used
- Supplies pairwise return relationships for calculating portfolio volatility and comparing portfolio choices.
- How deep it goes
- Derives covariance from correlation and volatility, implements matrix calculations, and estimates correlations from return histories.
- Examples
- Uses FE, WMT, and AAPL matrices and a stock–bond allocation slider with a negative correlation input.
- What you can do after
- Apply it via NumPy and pandas to estimate correlation matrices, form covariances, and calculate portfolio volatility.
Read how Build a Robo-Advisor with Python (From Scratch) explains correlation →
Outlier Detection in Python
Correlation is a measure of statistical association between paired numerical quantities, such as outputs from different detectors.
- How it is used
- Evaluates agreement and diversity among outlier detectors, emphasizing their highest-ranked records.
- How deep it goes
- Explains Pearson and Spearman measures, score filtering, ranking, and ensemble trade-offs, but does not derive correlation mathematically.
- Examples
- Uses abalone detector scores, top-score heatmaps, rank filtering, and PCA-transformed synthetic outliers as concrete examples.
- What you can do after
- Evaluate detector agreement with rank correlations and select more diverse ensembles without treating low correlation as sufficient evidence of quality.
Natural Language Processing in Action, Second Edition
A correlation coefficient is a numerical value quantifying the relationship between paired values in two sequences.
- How it is used
- Uses correlation to compare sequences, inspect learned representations, and measure local text-pattern matches.
- How deep it goes
- Defines Pearson normalization and demonstrates code, then applies correlation to PCA, RNN interpretation, and CNN sliding windows.
- Examples
- Demonstrates NumPy arrays, PCA feature dimensions, RNN hidden weights versus character positions, and CNN kernels over text windows.
- What you can do after
- Apply it via NumPy or scipy.stats to compare equal-length sequences and interpret relationships in learned numerical representations.
Read how Natural Language Processing in Action, Second Edition explains correlation →
Starting Data Analytics with Generative AI and Python
A statistical measure of the association between variables.
- How it is used
- Uses correlation to investigate variable relationships and support cautious interpretation of business patterns.
- How deep it goes
- Explains linear association, direction, strength, and major sources of misleading results, but does not derive the statistic.
- Examples
- Uses product, freight, payment, review-timestamp, delivery-time, and order-value relationships as practical examples.
- What you can do after
- Evaluate/understand relationships by reading direction and strength, inspecting plots, and checking whether time patterns or outliers distort conclusions.
Read how Starting Data Analytics with Generative AI and Python explains correlation →
Data-Driven Cybersecurity
A correlation is a statistical measure describing the strength and direction of association between variables.
- How it is used
- Identifies relationships among cybersecurity metrics to guide threat investigation and monitoring decisions.
- How deep it goes
- Defines strength and direction, interprets matrix values, and demonstrates a heatmap; stops short of deriving the calculation.
- Examples
- Uses failed logins, login hours, phishing emails, and a Python-generated correlation heatmap.
- What you can do after
- Apply it via pandas and seaborn to visualize security-metric relationships, then assess possible monitoring changes against context.
Bayesian Optimization in Action
Correlation is a numerical measure of the strength and direction of dependence between variables.
- How it is used
- Uses similarity between observations to update beliefs about an unknown quantity in Bayesian modeling and Gaussian processes.
- How deep it goes
- Defines correlation through multivariate Gaussian covariance matrices and illustrates its information-sharing effect, without deriving a scalar correlation formula.
- Examples
- Illustrates correlation with neighboring and distant house prices, multivariate Gaussian samples, and parallel-coordinate visualizations.
- What you can do after
- Evaluate how an observation’s similarity to a target affects the amount of information transferred in a Bayesian estimate.
Read how Bayesian Optimization in Action explains correlation →
Investing for Programmers
Correlation is a statistical measure of the strength and direction of the relationship between two variables, bounded between −1 and 1.
- How it is used
- Compares asset movements to support diversification and portfolio risk selection.
- How deep it goes
- Defines directional movement and computes return-based covariance and correlation matrices alongside variance and standard deviation.
- Examples
- Uses Microsoft, Amazon, Alphabet, Walmart, and NVIDIA return matrices to illustrate positive, near-zero, and comparative relationships.
- What you can do after
- Evaluate historical asset co-movement with Python data and correlation matrices when considering portfolio risk.
Statistics Every Programmer Needs
Correlation is a measure of the degree to which two quantities vary together in a linear association.
- How it is used
- Uses correlation to characterize linear associations and assess whether regression residuals remain independently distributed.
- How deep it goes
- Develops direction, scatter-plot interpretation, residual patterns, and Durbin–Watson testing within linear regression.
- Examples
- Uses positive, negative, and neutral scatter plots, regression-line data, residual plots, and a Durbin–Watson result.
- What you can do after
- Diagnose linear regression relationships and residual independence using plots and the Durbin–Watson test.
Read how Statistics Every Programmer Needs explains correlation →
Delivering Machine Learning Projects: From design to deployment
Pearson correlation is a standardized measure of the linear association between two numerical variables.
- How it is used
- Uses Pearson correlation in exploratory data analysis as a compact summary that must be checked against visual structure.
- How deep it goes
- Defines correlation through examples and stops short of mathematical derivation or algorithmic exposition.
- Examples
- Uses Anscombe’s Quartet and tips-versus-bills plots to expose misleading summaries and uneven dispersal.
- What you can do after
- Evaluate correlation by comparing its summary with plots before selecting or sampling modeling data.
Read how Delivering Machine Learning Projects: From design to deployment explains correlation →
Data Without Labels
Correlation is a statistical association describing how changes in one variable relate to changes in another variable.
- How it is used
- Uses correlation coefficients as filter methods for identifying redundant variables during dimensionality reduction.
- How deep it goes
- Defines mutual association and gives selection guidance, while deferring detailed correlation treatment to an appendix.
- Examples
- Uses retail invoices and sales, study hours and grades, and customer sales, invoices, and item counts.
- What you can do after
- Evaluate variable redundancy and cautiously select candidates for removal using correlation alongside business judgment.
Related concepts
- causationcontrasts with · Causation is a relationship in which one event, condition, or factor produces or influences a change in another.
- Spearman's rank correlation coefficientincludes · Spearman's rank correlation coefficient is a rank-based statistic that measures the strength and direction of the monotonic association between two variables.
- Pearson correlation coefficientincludes · The Pearson correlation coefficient measures the strength and direction of the linear relationship between two numerical variables.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.