MENTAL MODEL · M1758

Gradient Descent

TechnicalHigh supportAlgorithms

Included

Version 1.0.0 · Updated 2026-07-30

CORE DEFINITION

An iterative algorithm for finding a minimum. It is like descending a mountain in fog: although you cannot see the valley bottom, as long as you take a step in the steepest downward direction at each point, you will most likely reach the bottom eventually.

SCAFFOLDING EFFECT

psychology

Reduce cognitive load

- Action rule: When you do not know where the ultimate goal (valley bottom) is, do not stay put and overthink. As long as you ensure each step improves the current situation a little (following the gradient descent), iterate quickly, and you will eventually approach the optimal solution.

anchor

Anchor fast decisions

Iteratively update parameters along the negative gradient direction of the loss function to minimize error; the gradient points to the steepest ascent, so the opposite direction is the steepest descent. It is the engine of local search.

MINIMUM ACTION

In progress 0/5

Practice this model in one real situation:

Choose a loss function and learning rate
compute the gradient
update parameters by subtracting learning rate times gradient
iterate until convergence
adjust learning rate and avoid getting stuck in local minima.

Check to track your progress (stored locally)

Learning progress0%

account_treeGenealogyexpand_more

menu_bookReferencesexpand_more

Source support: Explicit

link
en.wikipedia.orghttps://en.wikipedia.org/wiki/Gradient_descentZH · Explicit
verified

RELATED MODELS