Artificial intelligence · 2020
GPT-3
175 billion parameters, and the model learns new tasks from examples in the prompt.
OpenAI's GPT-3, trained on about 300 billion tokens, showed in-context few-shot learning: give it a few examples and it performs the task without training. Released through an API, it started the commercial large language model industry.
What it built on 4
What it led to 7
Related measurements
The dashed line marks when it first worked.
Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)
ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)