Stone to Silicon

Artificial intelligence · 1997

LSTM

Gated memory cells let networks remember over thousands of steps.

Sepp Hochreiter and Juergen Schmidhuber's long short-term memory solved the vanishing gradient problem with gated cells that preserve information. By the 2010s LSTMs ran Google voice search, Apple's QuickType and Google Translate.

See it in the living treeWikipedia

Arcs above join it to what it stood on, arcs below to what it led to. The bar places this stretch on the tree's whole timeline.

What it built on 1

What it led to 3

Related measurements

The dashed line marks when it first worked.

Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)

Computation per constant dollar, 1850-2006 Latest: 7.3B million units of computer power per 2006 $ (2003) Source: Nordhaus 2007, Two Centuries of Productivity Growth in Computing, J. Econ. Hist. 67(1), Table 6