Cognitive Scaffold

Preparing your thinking workspace

arrow_back_ios_new
MENTAL MODEL · M0017

Vanishing Gradient

Vanishing Gradient
TechnicalHigh supportArtificial Intelligence
Included
account_tree

Version 1.0.0 · Updated 2026-07-30

CORE DEFINITION

Vanishing gradients occur when backpropagation through many layers or time steps produces very small gradients, slowing or stalling learning in affected parameters. This is not inkjet-printed circuit fabrication and does not imply that every deep network stops learning.

SCAFFOLDING EFFECT

psychology

Reduce cognitive load

Diagnose difficulties learning in early layers or across long time dependencies. Organizational information loss is only an analogy, not evidence for this mathematical mechanism.

anchor

Anchor fast decisions

In deep networks, during backpropagation, gradients are multiplied by coefficients less than 1 layer by layer, causing them to approach zero. As a result, earlier layers fail to learn, and training stalls. The chain product leads to attenuation.

MINIMUM ACTION

In progress 0/4

Practice this model in one real situation:

Check to track your progress (stored locally)
Learning progress0%
account_treeGenealogyexpand_more
menu_bookReferencesexpand_more

Source support: Explicit

  • link
    zh.wikipedia.orghttps://zh.wikipedia.org/wiki/%E6%A2%AF%E5%BA%A6%E6%B6%88%E5%A4%B1%E9%97%AE%E9%A2%98ZH · Explicit
    verified
  • link
    en.wikipedia.orghttps://en.wikipedia.org/wiki/Vanishing_gradient_problemEN
    verified

RELATED MODELS

Related model

Self-Attention Mechanism

Generates queries, keys, and values from the same sequence and aggregates representations using position-wise compatibility; masks and other constraints determine which positions are visible.

arrow_forwardRelated model

Maes-Garreau Law

The Maes–Garreau law is the statement that "most favorable predictions about future technology will fall within the Maes–Garreau point", defined as "the latest possible date a prediction can come true".

arrow_forwardRelated model

Means-Ends Analysis

Means–ends analysis (MEA) is a problem solving technique used commonly in artificial intelligence (AI) for limiting search in AI programs. MEA was designed by two scientist Allen Newel and Herbert A.

arrow_forward