Multiple Hypothesis Testing
Updated 2026-08-15
INTRODUCTION
English translation pending.
CORE DEFINITION
When many hypotheses are tested at once, the chance of at least one false positive grows with the number of tests, so a nominal significance threshold no longer controls error at its stated level, and with a thousand tests a significant result becomes almost certain by chance alone. Statistical practice therefore adjusts either the family-wise error rate, as in the Bonferroni correction, or the false discovery rate. The qualification is that adjustment trades power for control, so the choice depends on whether a single false claim or a mixed set of claims is more costly.
SCAFFOLDING EFFECT
Reduce cognitive load
- Use Result Skepticism: ask how many comparisons were run before believing a single significant finding. - Use Threshold Selection: choose family-wise or false discovery control based on the cost of a false claim.
Anchor fast decisions
Each test carries its own small error probability, and those probabilities accumulate across comparisons, so with enough tests a significant result becomes almost certain by chance alone. Adjustment raises the bar per test so that the combined error rate returns to the intended level.
MINIMUM ACTION
In progress 0/1Practice this model in one real situation:
account_treeGenealogyexpand_more
menu_bookReferencesexpand_more
Source support: Explicit
- en.wikipedia.orghttps://en.wikipedia.org/wiki/Multiple_comparisons_problemverified
PRIVATE NOTES · Only visible to you
SAVED Q&A
ENTRY Q&A · Private saving available
Ask with a clear boundary
thinkingmodels answers from published entry context only.
Your question is sent to thinkingmodels. The answer uses public entry context only.
RELATED MODELS