Artificial intelligence · 2006

Deep belief networks

Hinton shows deep networks can be trained by stacking layers one at a time.

Geoffrey Hinton, Simon Osindero and Yee-Whye Teh pre-trained deep networks layer by layer as restricted Boltzmann machines, then fine-tuned. The result broke the belief that deep nets could not be trained and rebranded the field as deep learning.

See it in the living tree Wikipedia

Arcs above join it to what it stood on, arcs below to what it led to. The bar places this stretch on the tree's whole timeline.

What it built on 2

Backpropagation 1986
Boltzmann machine 1985

What it led to 3

GPU-trained neural nets 2009
Deep speech recognition 2012
Google Brain cat neuron 2012

Related measurements

The dashed line marks when it first worked.

Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)

ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)