Artificial intelligence · 2022
InstructGPT
RLHF makes a 1.3B model preferred over the 175B GPT-3.
OpenAI fine-tuned GPT-3 on human demonstrations and then with reinforcement learning from human preference rankings. Labelers preferred the aligned 1.3-billion-parameter model's outputs to raw 175-billion-parameter GPT-3. This recipe became ChatGPT.
What it built on 2
What it led to 3
Related measurements
The dashed line marks when it first worked.
Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)
ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)