Understanding statistical power and how to use it effectively in your research design
Statistical power is the probability that your study will detect an effect when there actually is one. It's essentially your study's ability to avoid a Type II error (false negative).
Running an underpowered study is like looking for something important with poor lighting—you might miss what you're looking for even if it's there. This wastes resources, participant time, and can lead to misleading null findings.
The Standard: 80% Power
By convention, researchers aim for 80% power (β = 0.20). This means you have an 80% chance of detecting a true effect. Some fields or critical studies require 90% or 95% power.
The most challenging part of power analysis is estimating the expected effect size. Here are strategies:
Look for similar studies in your field. Extract means, standard deviations, and group sizes to calculate Cohen's d or other effect sizes. Meta-analyses are especially valuable as they aggregate multiple studies.
Run a small pilot study (n=10-20 per group) to estimate effect sizes. Be cautious: pilot studies often overestimate effects. Consider using 50-70% of the observed pilot effect for your power calculation.
As a last resort, use Cohen's conventional effect sizes:
Note: These are rules of thumb. Field-specific norms may differ.
Ask: "What's the smallest effect that would be practically meaningful?" For example, a 5-point improvement on a depression scale might be clinically significant. Calculate the effect size based on this threshold.
Some researchers justify small samples by calling studies "exploratory." But even exploratory work needs adequate power—otherwise you're just generating noise.
Solution: Frame underpowered pilots honestly as hypothesis-generating, not hypothesis-testing.
Calculating power after finding a non-significant result is circular reasoning. If p > 0.05, power will always be low. This tells you nothing useful.
Solution: Power analysis is for planning, not interpreting null results.
Publication bias means published effects are often larger than true effects. Using these directly will underpower your study.
Solution: Deflate literature estimates by 25-50%, or focus on meta-analytic estimates which are more conservative.
If 20% of participants drop out, your final sample will be smaller than planned, reducing power.
Solution: Add 10-20% to your target sample size to account for expected dropout.
α = 0.05 is convention, but exploratory studies might use 0.10, while high-stakes studies (e.g., clinical trials) might use 0.01.
Solution: Justify your alpha level based on study goals and consequences of errors.
Scenario: Testing if a new therapy reduces anxiety compared to standard care.
Effect Size Estimate: Meta-analysis shows similar therapies have d = 0.45.
Power Calculation: For 80% power at α = 0.05, you need approximately 80 participants per group (160 total).
Scenario: Investigating the relationship between sleep quality and academic performance.
Effect Size Estimate: Previous research suggests r = 0.30 (medium correlation).
Power Calculation: For 80% power at α = 0.05, you need approximately 85 participants.
Scenario: Comparing three teaching methods on test scores.
Effect Size Estimate: Pilot data suggests η² = 0.10 (medium effect).
Power Calculation: For 80% power at α = 0.05, you need approximately 53 participants per group (159 total).
For more research planning tools, you might find structuredvalidator.com helpful for complementary statistical analysis needs.
Use our interactive calculators to plan your research design with confidence.