Exploration-Exploitation Trade-off

CORE DEFINITION

The trade-off is formalized in the multi-armed bandit problem and is central to reinforcement learning and decision theory. Exploration means choosing an option whose value is uncertain in order to learn more; exploitation means choosing the option with the best known payoff. The core claim is that both pure strategies are suboptimal: pure exploitation locks in a possibly inferior option, while pure exploration never converts learning into return. The qualification is that the optimum depends on the horizon and on how fast the environment changes, which is why epsilon-greedy, upper confidence bound, and Gittins indices produce different answers rather than one universal rule.