Artificial intelligence · 2014

Sequence to sequence

An encoder LSTM reads a sentence and a decoder LSTM writes its translation.

Ilya Sutskever, Oriol Vinyals and Quoc Le showed that a pure neural encoder-decoder could translate English to French competitively with phrase-based systems. The encoder-decoder pattern became the template for translation, summarisation and speech.

See it in the living tree Wikipedia

Arcs above join it to what it stood on, arcs below to what it led to. The bar places this stretch on the tree's whole timeline.

What it built on 2

word2vec 2013
LSTM 1997

What it led to 3

Attention mechanism 2014
Google Neural Machine Translation 2016
Retrieval-augmented generation 2020

Related measurements

The dashed line marks when it first worked.

Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)

ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)