MENTAL MODEL · M3885

Inverse Scaling

StructuremediumLogic

Included

Updated 2026-08-11

Loading revision record…

INTRODUCTION

English translation pending.

CORE DEFINITION

Identified in the Inverse Scaling Prize run by McKenzie and colleagues, which collected tasks where performance degrades as model size grows. The cause is that larger models more faithfully reproduce a misleading pattern present in the training data. The claim is task-specific: it describes cases where the learned heuristic is wrong, not a general failure of scale.

SCAFFOLDING EFFECT

psychology

Reduce cognitive load

- Growth trap check: Look for domains where expansion makes results worse rather than better. - Assumption test: Question the linear belief that more resources always improve output. - Bottleneck hunt: Treat declining performance under scale as a signal of flawed underlying logic.

anchor

Anchor fast decisions

Scaling increases a model's ability to fit patterns in its training data, including patterns that are wrong. A small model lacks the capacity to reproduce the misleading heuristic and so avoids the error by default. Growth therefore amplifies whichever regularity is most represented, correct or not.

MINIMUM ACTION

In progress 0/1

Practice this model in one real situation:

Pick a task where you assume more resources will help. 2. Measure performance at a small scale and at a large one. 3. If the large scale is worse, look for a biased pattern being amplified. 4. Fix the underlying assumption rather than adding resources. 5. Re-measure before scaling any further.

Check to track your progress (stored locally)

Learning progress0%

account_treeGenealogyexpand_more

menu_bookReferencesexpand_more

Source support: Explicit

link
arxiv.orghttps://arxiv.org/abs/2306.09479ZH · Explicit
verified

RELATED MODELS