Power analysis
Power analysis is a statistical method for estimating a test’s ability to detect an effect under specified assumptions, often to determine an appropriate sample size.
Effective Data Analysis
An a priori power analysis determines the sample size required to achieve a desired probability of detecting an effect of a specified magnitude at a chosen significance level.
- How it is used
- Plans sample collection before t-tests or ANOVAs and supports defensible decisions about statistical evidence.
- How deep it goes
- Explains parameter choices, assumption checks, conservative targets, and implementation, but does not derive the underlying formulas.
- Examples
- Uses t-test simulations, Boston–New York weather data, and a four-group ANOVA calculation as concrete applications.
- What you can do after
- Apply it via statsmodels to estimate minimum sample sizes and plan appropriately powered statistical tests.
Experimentation for Engineers
Power analysis is the calculation of the sample size or sensitivity needed to limit the probability of failing to detect a specified effect.
- How it is used
- Designs A/B tests by choosing enough measurements to detect a practically significant change in a business metric.
- How deep it goes
- Derives the sample-size inequality and implements it numerically, but does not develop a general power-analysis theory.
- Examples
- Uses simulated trading systems, execution-cost variability, practical significance of 1.0 mip, and seven-versus-sixteen-measurement designs.
- What you can do after
- Build it from scratch by estimating variability, selecting a meaningful effect, and calculating a rounded-up measurement count.
Read how Experimentation for Engineers explains power analysis →
Hacking Cryptography
Power analysis is a side-channel technique that infers computational behavior or secrets from a device's power consumption.
- How it is used
- Infers private-exponent bits from power differences during cryptographic exponentiation.
- How deep it goes
- Traces an oscilloscope-based attack and contrasts operation leakage with constant-time implementation, stopping short of optimization details.
- Examples
- Uses RSA power traces, square-and-multiply for exponent 42, and the Montgomery powering ladder.
- What you can do after
- Apply it via an oscilloscope to analyze operation-dependent power traces and recognize constant-operation defenses.
AI Model Evaluation
Power analysis is a statistical procedure for determining the sample size needed to detect an effect under specified assumptions.
- How it is used
- Uses baseline rates and minimum detectable effects to plan sample sizes before launching model A/B tests.
- How deep it goes
- Defines the planning inputs and timing but does not explain the statistical calculation or power-analysis formula.
- Examples
- Applies planning to a movie recommendation test with user-level randomization, eligible users, and a four-week duration.
- What you can do after
- Operate/configure an A/B test by replacing generic sample-size rules with an estimate tied to baseline rates and detectable effects.
What to know first
- private keyA private key is secret cryptographic key material associated with a corresponding public key and used for operations such as signing or decryption.
Related concepts
- sample sizeproduces · Sample size is the number of observations, units, or items included in a sample or group being analyzed.
- side-channel attackused by · A side-channel attack infers secret information from unintended physical or operational leakage produced by a system’s implementation rather than its intended outputs.
- effect sizeuses · An effect size is a quantitative measure of the magnitude of a difference, association, relationship, or other observed phenomenon, independent of statistical significance.
- statistical poweruses · Statistical power is the probability that a statistical test detects a specified effect when that effect is present.
- study designpart of · Study design is the planned arrangement of procedures, conditions, measurements, and comparisons for investigating a question and supporting valid conclusions.
Questions about power analysis
What is power analysis?
Power analysis is a statistical method for estimating a test’s ability to detect an effect under specified assumptions, often to determine an appropriate sample size.
Which Manning books explain power analysis?
Effective Data Analysis, Experimentation for Engineers, Hacking Cryptography and AI Model Evaluation.
What should I know before learning power analysis?
Manning books that teach power analysis usually introduce private key first.
What is power analysis part of?
Power analysis is part of study design. Study design is the planned arrangement of procedures, conditions, measurements, and comparisons for investigating a question and supporting valid conclusions.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



