Artificial intelligence · 2020

MMLU benchmark

57 subjects of exam questions become the scoreboard for language models.

Dan Hendrycks and colleagues released Measuring Massive Multitask Language Understanding, about 16,000 multiple-choice questions from law to physics. GPT-3 scored about 44%. Frontier models passed 85% by 2023 and effectively saturated it by 2024.

See it in the living tree Wikipedia

Arcs above join it to what it stood on, arcs below to what it led to. The bar places this stretch on the tree's whole timeline.

What it built on 1

BERT 2018

What it led to 1

Humanity's Last Exam 2025

Related measurements

The dashed line marks when it first worked.

Best AI score on GPQA Diamond Latest: 95.8 % correct (2026) Source: Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv); Anthropic: Claude 3 public launch (March 4, 2024)

Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)