MENTAL MODEL · M2311

Automation Bias

DecideHigh supportDecision Science

Included

Version 1.0.0 · Updated 2026-07-30

CORE DEFINITION

Humans tend to over-trust the recommendations provided by automated systems (algorithms, dashboards, AI), even when there is clear evidence that the system is wrong, and they choose to trust the system over their own judgment. -

SCAFFOLDING EFFECT

psychology

Reduce cognitive load

An alarm against blind obedience in the AI era. - Pilots who trust the dashboard and ignore the mountain outside the window, doctors who trust AI diagnoses and ignore patient symptoms. In the era of AI-assisted decision-making, 'maintaining independent judgment with a human in the loop' will become a scarce safety capability. Never hand the 'final confirmation button' to the algorithm.

anchor

Anchor fast decisions

Over-trust in automated advice, even when there is counter-evidence, leading to compliance and neglect of one's own judgment. Its roots lie in cognitive economy (saving effort) and the illusion of 'machine objectivity', as well as the delegation of attention to the system under multi-task load. It weakens independent monitoring willingness, forming 'automation complacency'.

MINIMUM ACTION

In progress 0/3

Practice this model in one real situation:

Maintain independent judgment with a 'human in the loop' in AI-assisted decision-making: set verification steps and explainable evidence for system outputs
train to identify false positives and false negatives
keep the final confirmation authority with humans. Regularly conduct red-team exercises assuming 'the system may be wrong' to prevent alertness dulling.

Check to track your progress (stored locally)

Learning progress0%

account_treeGenealogyexpand_more

menu_bookReferencesexpand_more

Source support: Explicit

link
en.wikipedia.orghttps://en.wikipedia.org/wiki/Automation_biasZH · Explicit
verified

RELATED MODELS