Artificial intelligence · 1986
Backpropagation
The chain rule, applied backwards, finally trains networks with hidden layers.
David Rumelhart, Geoffrey Hinton and Ronald Williams showed in Nature that backpropagating errors lets multilayer networks learn useful internal representations. Earlier derivations existed (Linnainmaa 1970, Werbos 1974), but this paper launched the connectionist revival. Every modern neural network is trained this way.
What it built on 3
What it led to 7
Related measurements
The dashed line marks when it first worked.
Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)
Computation per constant dollar, 1850-2006 Latest: 7.3B million units of computer power per 2006 $ (2003) Source: Nordhaus 2007, Two Centuries of Productivity Growth in Computing, J. Econ. Hist. 67(1), Table 6