Artificial intelligence · 1990

Recurrent neural networks

Networks with loops carry memory forward to process sequences like language.

Jeffrey Elman's simple recurrent network fed hidden state back as input, learning grammatical structure from word sequences. Trained by backpropagation through time, RNNs became the default for speech and text until transformers, though they struggled to remember far back.

See it in the living treeWikipedia