Artificial intelligence · 2017
Proximal policy optimization
A simple, stable policy gradient method that becomes RL's default.
John Schulman and colleagues at OpenAI introduced PPO, clipping updates to stay near the current policy. It trained OpenAI Five and was the reinforcement step in InstructGPT and early ChatGPT.
What it built on 1
What it led to 3
Related measurements
The dashed line marks when it first worked.
Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)
ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)