Correlation
Correlation is a statistical association describing how variables vary together in direction or degree, without by itself establishing causation.
Pandas Workout: 200 exercises to make you a stronger data analyst
A statistical measure of the strength and direction of association between the number of wines submitted by a country and its mean score.
- How it is used
- Uses correlation to investigate associations in grouped, joined, and cleaned real-world datasets.
- How deep it goes
- Defines direction and strength, interprets values from –1 to 1, and applies `corr()` alongside scatter plots.
- Examples
- Uses temperatures, oil prices, ice-cream prices, miles driven, wine scores, tourism spending, tuition, and admission rates.
- What you can do after
- Apply it via pandas `corr()` to quantify numeric associations and inspect them with scatter plots.
Read how Pandas Workout: 200 exercises to make you a stronger data analyst explains correlation →
Effective Data Analysis
Correlation is a statistical association describing how two variables vary together, without by itself establishing causation.
- How it is used
- Uses correlations to explore hypotheses and identify candidate relationships for predictive modeling and stakeholder decisions.
- How deep it goes
- Compares coefficient types, interpretation, assumptions, and visualization, with a worked matrix and guidance for reporting.
- Examples
- Examines daily rat sightings alongside weather and assigns an exercise exploring correlations with product profit margins.
- What you can do after
- Apply it via pandas to explore associations and communicate their direction, strength, and limitations.
Statistics Slam Dunk
A correlation coefficient is a numeric measure of the direction and strength of the linear relationship between two variables, ranging from -1 to +1.
- How it is used
- Screens numeric NBA variables for associations and potential linear-regression predictors.
- How deep it goes
- Defines coefficient interpretation, computes matrices and tests, and visualizes scatterplots without deriving the statistic.
- Examples
- Uses draft statistics, salaries and win shares, team payrolls and wins, and points scored or allowed versus wins.
- What you can do after
- Apply it via base R cor() and ggplot2 to compute, visualize, and compare numeric-variable associations.
Causal Inference for Data Science
Correlation is a statistical association in which two variables vary together without by itself establishing that either causes the other.
- How it is used
- Distinguishes observed association from evidence needed for causal decisions and supports predictive counterfactuals.
- How deep it goes
- Develops common-cause intuition, synthetic simulations, graph-based path calculations, and conditioning effects in linear models.
- Examples
- Uses city health, arcade revenue and doctorates, simulated population data, and paired movie-interest time series.
- What you can do after
- Evaluate/understand why an observed association may be confounded and how graph paths determine correlation in normalized linear models.
Read how Causal Inference for Data Science explains correlation →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
Bivariate correlation is a statistical measure of association between two variables or features.
- How it is used
- Screens numeric tabular features for redundancy and collinearity before feature selection or modeling.
- How deep it goes
- Explains pairwise matrices, thresholds, and feature-removal decisions, while contrasting correlation with broader VIF analysis.
- Examples
- Demonstrates synthetic near-duplicate numeric features and a pandas correlation-matrix seaborn heatmap.
- What you can do after
- Apply it via pandas and seaborn to flag correlated numeric-feature pairs for investigation and possible removal.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains correlation →
Grokking Statistics
A correlation coefficient is a numerical measure summarizing the strength and direction of association between two variables.
- How it is used
- Uses correlation to investigate relationships between paired variables before prediction with linear regression.
- How deep it goes
- Defines directional and linear association, interprets Pearson r, and demonstrates calculation, while deferring formula proofs.
- Examples
- Uses sleep and happiness, social media and happiness, rainfall, temperature and sports-drink sales, and spurious pool-drowning data.
- What you can do after
- Evaluate/understand Pearson correlation values, inspect linear patterns, and resist causal conclusions from association alone.
Build a Robo-Advisor with Python (From Scratch)
Correlation is a statistical measure of the direction and strength of dependence between two variables or asset returns.
- How it is used
- Supplies pairwise return relationships for calculating portfolio volatility and comparing portfolio choices.
- How deep it goes
- Derives covariance from correlation and volatility, implements matrix calculations, and estimates correlations from return histories.
- Examples
- Uses FE, WMT, and AAPL matrices and a stock–bond allocation slider with a negative correlation input.
- What you can do after
- Apply it via NumPy and pandas to estimate correlation matrices, form covariances, and calculate portfolio volatility.
Read how Build a Robo-Advisor with Python (From Scratch) explains correlation →
Outlier Detection in Python
Correlation is a measure of statistical association between paired numerical quantities, such as outputs from different detectors.
- How it is used
- Evaluates agreement and diversity among outlier detectors, emphasizing their highest-ranked records.
- How deep it goes
- Explains Pearson and Spearman measures, score filtering, ranking, and ensemble trade-offs, but does not derive correlation mathematically.
- Examples
- Uses abalone detector scores, top-score heatmaps, rank filtering, and PCA-transformed synthetic outliers as concrete examples.
- What you can do after
- Evaluate detector agreement with rank correlations and select more diverse ensembles without treating low correlation as sufficient evidence of quality.
Timeless Algorithms, The Foundational Ideas
Correlation measures the strength and direction of association between quantities.
- How it is used
- Explains why aggregation benefits from trees whose prediction errors do not repeatedly align.
- How deep it goes
- Defines correlation through classification margin behavior and relates it to strength in a generalization-error bound.
- Examples
- Uses correlated tree errors and a pedestrian detector’s reliance on lighting or background to illustrate different consequences of association.
- What you can do after
- Evaluate and understand how correlation limits ensemble gains and how learned associations may fail under distribution shift.
Read how Timeless Algorithms, The Foundational Ideas explains correlation →
Natural Language Processing in Action, Second Edition
A correlation coefficient is a numerical value quantifying the relationship between paired values in two sequences.
- How it is used
- Uses correlation to compare sequences, inspect learned representations, and measure local text-pattern matches.
- How deep it goes
- Defines Pearson normalization and demonstrates code, then applies correlation to PCA, RNN interpretation, and CNN sliding windows.
- Examples
- Demonstrates NumPy arrays, PCA feature dimensions, RNN hidden weights versus character positions, and CNN kernels over text windows.
- What you can do after
- Apply it via NumPy or scipy.stats to compare equal-length sequences and interpret relationships in learned numerical representations.
Read how Natural Language Processing in Action, Second Edition explains correlation →
Starting Data Analytics with Generative AI and Python
A statistical measure of the association between variables.
- How it is used
- Uses correlation to investigate variable relationships and support cautious interpretation of business patterns.
- How deep it goes
- Explains linear association, direction, strength, and major sources of misleading results, but does not derive the statistic.
- Examples
- Uses product, freight, payment, review-timestamp, delivery-time, and order-value relationships as practical examples.
- What you can do after
- Evaluate/understand relationships by reading direction and strength, inspecting plots, and checking whether time patterns or outliers distort conclusions.
Read how Starting Data Analytics with Generative AI and Python explains correlation →
Data-Driven Cybersecurity
A correlation is a statistical measure describing the strength and direction of association between variables.
- How it is used
- Identifies relationships among cybersecurity metrics to guide threat investigation and monitoring decisions.
- How deep it goes
- Defines strength and direction, interprets matrix values, and demonstrates a heatmap; stops short of deriving the calculation.
- Examples
- Uses failed logins, login hours, phishing emails, and a Python-generated correlation heatmap.
- What you can do after
- Apply it via pandas and seaborn to visualize security-metric relationships, then assess possible monitoring changes against context.
Bayesian Optimization in Action
Correlation is a numerical measure of the strength and direction of dependence between variables.
- How it is used
- Uses similarity between observations to update beliefs about an unknown quantity in Bayesian modeling and Gaussian processes.
- How deep it goes
- Defines correlation through multivariate Gaussian covariance matrices and illustrates its information-sharing effect, without deriving a scalar correlation formula.
- Examples
- Illustrates correlation with neighboring and distant house prices, multivariate Gaussian samples, and parallel-coordinate visualizations.
- What you can do after
- Evaluate how an observation’s similarity to a target affects the amount of information transferred in a Bayesian estimate.
Read how Bayesian Optimization in Action explains correlation →
Build AI Drug Discovery Pipelines
The Pearson correlation coefficient is a statistic ranging from -1 to 1 that measures the strength and direction of a linear association between two continuous variables.
- How it is used
- Uses Pearson correlation to inspect molecular features, guide feature engineering, and evaluate predicted biological quantities.
- How deep it goes
- Defines the coefficient, interprets its range, and applies it to descriptors and model predictions without deriving the statistic.
- Examples
- Uses logP, hydrogen-bond donors, TPSA, solubility, DGraphDTA predictions, and Davis and KIBA affinity datasets.
- What you can do after
- Evaluate linear associations and model ranking quality with Pearson correlation alongside error metrics.
Read how Build AI Drug Discovery Pipelines explains correlation →
Investing for Programmers
Correlation is a statistical measure of the strength and direction of the relationship between two variables, bounded between −1 and 1.
- How it is used
- Compares asset movements to support diversification and portfolio risk selection.
- How deep it goes
- Defines directional movement and computes return-based covariance and correlation matrices alongside variance and standard deviation.
- Examples
- Uses Microsoft, Amazon, Alphabet, Walmart, and NVIDIA return matrices to illustrate positive, near-zero, and comparative relationships.
- What you can do after
- Evaluate historical asset co-movement with Python data and correlation matrices when considering portfolio risk.
AI Model Evaluation
Correlation is a statistical association between variables that does not by itself establish that one variable causes another.
- How it is used
- Connects offline model metrics to online product outcomes during repeated evaluation and model-selection decisions.
- How deep it goes
- Explains directional strength and a rolling measurement process, but does not establish causation or specify a complete statistical method.
- Examples
- Uses precision@10, recall, diversity, click-through rate, watch time, retention, movie recommendations, and repeated A/B tests.
- What you can do after
- Diagnose whether offline metric improvements reliably align with online outcomes using a maintained historical dataset.
LLM Evaluation and Alignment, The Foundational Ideas
A correlation coefficient is a numerical measure of the consistency of the relationship or ordering between two sets of scores, independent of their absolute scales.
- How it is used
- Compares automated or LLM-based evaluator rankings with human preferences for model-output evaluation.
- How deep it goes
- Defines rank consistency, distinguishes it from categorical agreement, and works through a five-response example.
- Examples
- Uses customer-service chatbot ratings from a human expert and an LLM judge across five responses.
- What you can do after
- Evaluate whether an automated metric orders outputs consistently with human judgments.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains correlation →
Statistics Every Programmer Needs
Correlation is a measure of the degree to which two quantities vary together in a linear association.
- How it is used
- Uses correlation to characterize linear associations and assess whether regression residuals remain independently distributed.
- How deep it goes
- Develops direction, scatter-plot interpretation, residual patterns, and Durbin–Watson testing within linear regression.
- Examples
- Uses positive, negative, and neutral scatter plots, regression-line data, residual plots, and a Durbin–Watson result.
- What you can do after
- Diagnose linear regression relationships and residual independence using plots and the Durbin–Watson test.
Read how Statistics Every Programmer Needs explains correlation →
Hidden Influences
A statistical association in which two variables vary together without by itself establishing that one causes the other.
- How it is used
- Interprets relationships between social-media exposure and users’ later emotional expression without claiming causation.
- How deep it goes
- Defines correlation, distinguishes it from causation, and examines effect size and sentiment-measurement limitations.
- Examples
- Uses positive and negative social-media posts, a B=−0.07% effect, and the phrase “not a joyful day.”
- What you can do after
- Evaluate observed associations by separating correlation from causation and checking effect size and measurement quality.
Claude at Work
Correlation is a statistical association describing how two variables vary together.
- How it is used
- Uses correlation as a question for exploring relationships across public datasets.
- How deep it goes
- Frames a data exploration around rigorous methods but gives no calculation or interpretation of correlation.
- Examples
- Compares internet use, R&D researchers per million people, and annual space launches.
- What you can do after
- Evaluate possible associations by asking Claude to explore downloaded datasets and produce a research report.
Delivering Machine Learning Projects: From design to deployment
Pearson correlation is a standardized measure of the linear association between two numerical variables.
- How it is used
- Uses Pearson correlation in exploratory data analysis as a compact summary that must be checked against visual structure.
- How deep it goes
- Defines correlation through examples and stops short of mathematical derivation or algorithmic exposition.
- Examples
- Uses Anscombe’s Quartet and tips-versus-bills plots to expose misleading summaries and uneven dispersal.
- What you can do after
- Evaluate correlation by comparing its summary with plots before selecting or sampling modeling data.
Read how Delivering Machine Learning Projects: From design to deployment explains correlation →
Data Without Labels
Correlation is a statistical association describing how changes in one variable relate to changes in another variable.
- How it is used
- Uses correlation coefficients as filter methods for identifying redundant variables during dimensionality reduction.
- How deep it goes
- Defines mutual association and gives selection guidance, while deferring detailed correlation treatment to an appendix.
- Examples
- Uses retail invoices and sales, study hours and grades, and customer sales, invoices, and item counts.
- What you can do after
- Evaluate variable redundancy and cautiously select candidates for removal using correlation alongside business judgment.
Also covered in
- Privacy-Preserving Machine Learning
- Causal AI
- The Coder CafeIn early access (MEAP)
- Cyber Threat Hunting
- Machines that ThinkIn early access (MEAP)
- Evaluating AI SystemsIn early access (MEAP)
- Math and Architectures of Deep Learning
- Financial AI in Practice
What to know first
- machine learningMachine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
- data frameA data frame is a two-dimensional tabular data structure organized into labeled rows and columns, often with columns that can contain different types of values.
- varianceVariance is the expected squared deviation of a random variable from its mean and measures the dispersion of its values.
- linear regressionLinear regression is a statistical method that models a numeric response as a linear function of one or more predictor variables with fitted parameters.
Related concepts
- causationcontrasts with · Causation is a relationship in which a change in one event, factor, or condition produces or influences a change in another.
- Spearman's rank correlation coefficientincludes · Spearman's rank correlation coefficient is a rank-based statistic measuring the strength and direction of the monotonic association between two variables.
- Kendall's tauincludes · Kendall’s tau is a rank-correlation coefficient that measures ordinal association between paired observations by comparing concordant and discordant pairs.
- Pearson correlation coefficientincludes · The Pearson correlation coefficient is a standardized measure of the strength and direction of the linear association between two numerical variables.
Questions about correlation
What is correlation?
Correlation is a statistical association describing how variables vary together in direction or degree, without by itself establishing causation.
Which Manning books explain correlation?
Pandas Workout: 200 exercises to make you a stronger data analyst, Effective Data Analysis, Statistics Slam Dunk, Causal Inference for Data Science, Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI, Grokking Statistics, Build a Robo-Advisor with Python (From Scratch), Outlier Detection in Python and 14 more.
What should I know before learning correlation?
Manning books that teach correlation usually introduce machine learning, data frame, variance and linear regression first.
What is the difference between correlation and causation?
Correlation is a statistical association describing how variables vary together in direction or degree, without by itself establishing causation. Causation is a relationship in which a change in one event, factor, or condition produces or influences a change in another.
What kinds of correlation are there?
Kinds of correlation include Spearman's rank correlation coefficient, Kendall's tau and Pearson correlation coefficient.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.





















