MENTAL MODEL · M13019

Confidence Calibration

SystemsHigh supportMetacognition

Included

Updated 2026-08-11

Loading revision record…

INTRODUCTION

English translation pending.

CORE DEFINITION

Confidence calibration is the degree to which a person's subjective confidence matches objective accuracy: a well-calibrated judge who says eighty per cent is right about eighty per cent of the time. Philip Tetlock's long-run study of expert political forecasting, published as Expert Political Judgment, found that the most famous and most confident experts were among the least accurate. Weather forecasters are the standard counter-example, because they publish explicit probabilities and are scored against outcomes immediately.

SCAFFOLDING EFFECT

psychology

Reduce cognitive load

- Numeric commitment: replace words like likely with an explicit probability before deciding - Personal ledger: record the forecast and the outcome so frequencies can be computed - Bucket review: group past forecasts by probability band and check the realised hit rate against your stated confidence

anchor

Anchor fast decisions

Confidence is an internal feeling, while accuracy can only be measured against external outcomes, and the two have no necessary connection, so confidence generally runs above the actual hit rate. Calibration improves judgement through a feedback loop: when judgements are written in checkable form and results are recorded, the subjective feeling is gradually corrected by the observed frequency. In domains without prompt feedback, such as long-horizon political prediction, that corrective pressure never arrives and overconfidence can persist indefinitely.

MINIMUM ACTION

In progress 0/1

Practice this model in one real situation:

Attach an explicit probability to every forecast instead of using vague likelihood words. 2. Record the forecast and its outcome in a personal ledger. 3. Periodically compute the realised hit rate for each probability band. 4. For domains without feedback, set a date at which the claim gets checked.

Check to track your progress (stored locally)

Learning progress0%

account_treeGenealogyexpand_more

menu_bookReferencesexpand_more

Source support: Explicit

link
mungermodels.comhttps://mungermodels.com/models/confidence-calibrationZH · Explicit
verified

RELATED MODELS