Artificial intelligence · 1989

Q-learning

An agent learns the value of every action in every state with no model of the world.

Christopher Watkins introduced Q-learning in his 1989 PhD thesis, with a convergence proof following in 1992. It is off-policy and model-free, and in 2013 it became the core of DeepMind's DQN.

See it in the living tree Wikipedia

Arcs above join it to what it stood on, arcs below to what it led to. The bar places this stretch on the tree's whole timeline.

What it built on 1

Temporal-difference learning 1988

What it led to 1

Deep Q-network (Atari) 2013

Related measurements

The dashed line marks when it first worked.

Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)

Computation per constant dollar, 1850-2006 Latest: 7.3B million units of computer power per 2006 $ (2003) Source: Nordhaus 2007, Two Centuries of Productivity Growth in Computing, J. Econ. Hist. 67(1), Table 6