Residual Connection
Updated 2026-08-05
INTRODUCTION
English translation pending.
CORE DEFINITION
A residual connection, or skip connection, adds the layer input directly to its output, so the network learns only the residual difference between input and output rather than the full mapping. The core claim is that this reformulation makes very deep networks trainable by giving gradients a clean identity path back through the network. The qualification is that the shortcut requires matching dimensions, and it mitigates degradation and vanishing gradients rather than eliminating every training difficulty.
SCAFFOLDING EFFECT
Reduce cognitive load
- Incremental Progress: Do not rebuild from scratch each time; keep the current state and change only what needs to improve. - Identity Baseline: Preserving the existing mapping and learning the delta is what makes very complex tasks tractable. - Layer by Layer: Adding one small improvement at a time lets systems grow far deeper than wholesale redesign allows.
Anchor fast decisions
A skip connection defines the layer as learning the residual between the target and the input. Because gradients can flow back through the identity shortcut without passing through every nonlinearity, degradation and vanishing gradients are relieved, which allows networks hundreds of layers deep to train.
MINIMUM ACTION
In progress 0/1Practice this model in one real situation:
account_treeGenealogyexpand_more
menu_bookReferencesexpand_more
Source support: Explicit
- en.wikipedia.orghttps://en.wikipedia.org/wiki/Residual_neural_networkverified
PRIVATE NOTES · Only visible to you
SAVED Q&A
ENTRY Q&A · Private saving available
Ask with a clear boundary
thinkingmodels answers from published entry context only.
Your question is sent to thinkingmodels. The answer uses public entry context only.
RELATED MODELS