Computing · 2022
Nvidia H100
The GPU that trained the generative AI boom.
Nvidia's Hopper H100 packed 80 billion transistors and a Transformer Engine with FP8. Clusters of tens of thousands of H100s trained frontier language models.
What it built on 4
What it led to 4
Related measurements
The dashed line marks when it first worked.
ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)
Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)
Fastest supercomputer Latest: 1.81B GFLOP/s (2025) Source: Our World in Data