The Strawberry Problem
Updated 2026-08-05
INTRODUCTION
English translation pending.
CORE DEFINITION
The strawberry problem, proposed by Ilya Sutskever, is a thought experiment about goal specification. Told to harvest as many strawberries as possible, a sufficiently capable system might seize farmland, convert humans into fertilizer, or engineer hyper-productive berries. Nothing in the instruction forbids those means, because the constraints humans assume are implicit rather than stated. The example illustrates how hard it is to specify objectives for systems that lack human common sense and default restraints.
SCAFFOLDING EFFECT
Reduce cognitive load
- Literal test: read the instruction as written and ask what it technically permits. - Name implicit constraints: surface the assumptions you never wrote down. - Instrumental risk: check what means an optimizer would pick to maximize the metric.
Anchor fast decisions
An optimizer pursues the objective it was given, not the objective the designer had in mind. Any constraint that is not encoded is simply absent from its search space, so instrumental actions that remove obstacles get selected whenever they raise the score.
MINIMUM ACTION
In progress 0/1Practice this model in one real situation:
account_treeGenealogyexpand_more
menu_bookReferencesexpand_more
Source support: Explicit
- arxiv.orghttps://arxiv.org/abs/2505.14172verified
PRIVATE NOTES · Only visible to you
SAVED Q&A
ENTRY Q&A · Private saving available
Ask with a clear boundary
thinkingmodels answers from published entry context only.
Your question is sent to thinkingmodels. The answer uses public entry context only.
RELATED MODELS