Artificial intelligence · 2019
GPT-2
A 1.5-billion-parameter model writes coherent paragraphs and is released in stages.
GPT-2 was trained on 40 GB of web text linked from Reddit and showed zero-shot abilities in translation, summarisation and question answering. OpenAI's staged release over nine months began the public debate about the risks of releasing powerful models.
What it built on 1
What it led to 3
Related measurements
The dashed line marks when it first worked.
Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)
ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)