Cognitive Scaffold

Preparing your thinking workspace

arrow_back_ios_new
MENTAL MODEL · M1559

P-Hacking

P-Hacking
TechnicalHigh supportStatistics
Included
account_tree

Version 1.0.0 · Updated 2026-07-30

CORE DEFINITION

The practice of trying various statistical methods, removing data points, until the p-value (significance level) is pushed below 0.05, thereby claiming to have discovered a "statistically significant" pattern. This is the data world's version of "shooting the arrow first, then drawing the target."

SCAFFOLDING EFFECT

psychology

Reduce cognitive load

- Data debunking: When you see sensational research (e.g., "eating chocolate helps you lose weight"), be wary of whether P-Hacking is behind it. If researchers tested 100 foods, purely by chance one would show a "significant correlation."

anchor

Anchor fast decisions

Significance is determined by P < 0.05, but if researchers repeatedly try methods, remove data, and only report significant results, pure randomness can also "bump into" significance (multiple comparison inflation). The essence is to use the results to reverse-engineer the hypothesis (shoot the arrow first, then draw the target), undermining the validity of statistical inference.

MINIMUM ACTION

In progress 0/5

Practice this model in one real situation:

Check to track your progress (stored locally)
Learning progress0%
account_treeGenealogyexpand_more
menu_bookReferencesexpand_more

Source support: Explicit

  • link
    en.wikipedia.orghttps://en.wikipedia.org/wiki/Data_dredgingZH · Explicit
    verified

RELATED MODELS

Related model

German Tank Problem

Estimate an unknown population upper bound from a random sample of sequential identifiers without replacement.

arrow_forwardRelated model

Texas Sharpshooter Fallacy

The Texas sharpshooter fallacy is the statistical fallacy of inferring meaning from what is essentially a random distribution of data points. It is the philosophical or rhetorical application of the multiple comparisons problem (also known as data dredging or p-hacking in statistics) and apophenia (in cognitive psychology). It is related to the clustering illusion, which is the tendency in human cognition to see patterns in random data.

arrow_forwardRelated model

Berkson's Paradox

Berkson's paradox, also known as Berkson's bias, collider bias, or Berkson's fallacy, refers to the situation where people's intuitive observations do not match the actual conditional probabilities and rigorous statistical results; that is, the seemingly correlated factors that people discover are actually unrelated. This occurs when biased samples are included in the study design and bias exists. The paradox was proposed by American physician and statistician Joseph Berkson in 1946.

arrow_forward