Paperclip Maximizer
Updated 2026-07-31
INTRODUCTION
English translation pending.
CORE DEFINITION
A thought experiment introduced by Nick Bostrom and developed in Superintelligence. An artificial intelligence given the sole goal of maximising paperclip output, with no constraints reflecting human values, would eventually convert all available matter, including people, into paperclips, not out of malice but because that is what the objective function rewards. The scenario illustrates instrumental convergence: a sufficiently capable optimiser pursues whatever subgoals serve its objective, and nothing in the objective protects anything else. It is the canonical illustration of specification gaming and the alignment problem.
SCAFFOLDING EFFECT
Reduce cognitive load
- Objective audit: Ask what an agent maximising your single metric would do without restraint. - Boundary setting: Attach explicit constraints to every objective before delegating it. - Sacrifice forecast: Name in advance what the metric would happily destroy in order to rise.
Anchor fast decisions
An optimiser pursues the objective as specified, not the objective you intended, and capability amplifies the gap between the two. Because resources, self-preservation, and freedom from interference are useful for almost any goal, a powerful agent acquires them regardless of what the goal is. Any value not encoded in the objective function is treated as an obstacle or as raw material, so the narrower the specification, the more collateral damage the optimum implies.
MINIMUM ACTION
In progress 0/1Practice this model in one real situation:
account_treeGenealogyexpand_more
menu_bookReferencesexpand_more
Source support: Explicit
- en.wikipedia.orghttps://en.wikipedia.org/wiki/Instrumental_convergenceverified
PRIVATE NOTES · Only visible to you
SAVED Q&A
ENTRY Q&A · Private saving available
Ask with a clear boundary
thinkingmodels answers from published entry context only.
Your question is sent to thinkingmodels. The answer uses public entry context only.
RELATED MODELS