Artificial intelligence · 2009

GPU-trained neural nets

Graphics cards speed neural network training up to 70-fold.

Rajat Raina, Anand Madhavan and Andrew Ng showed that GPUs trained deep belief networks up to 70 times faster than CPUs. Dan Ciresan's GPU nets then won vision contests in 2011. Commodity GPUs became the engine of the deep learning era.

See it in the living tree Wikipedia

Arcs above join it to what it stood on, arcs below to what it led to. The bar places this stretch on the tree's whole timeline.

What it built on 3

CUDA 2007
Deep belief networks 2006
GPU (GeForce 256) 1999

What it led to 3

Deep learning (AlexNet) 2012
Deep speech recognition 2012
TensorFlow 2015

Related measurements

The dashed line marks when it first worked.

ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)

Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)