The Lottery Ticket Hypothesis

CORE DEFINITION

Proposed by Jonathan Frankle and Michael Carbin. They showed that a standard over-parameterised network contains subnetworks, called winning tickets, that can be trained in isolation to accuracy matching the full network, provided the surviving weights are reset to the values they held at initialisation. The implication is that dense training works largely as a search over a pool of candidate subnetworks. It also explains why pruning and then rewinding, rather than reinitialising, is what preserves performance. The claim is empirical and architecture-dependent, not a guarantee that every task hides a tiny sufficient network.