Computing · 2017
Tensor Cores (Volta)
Dedicated matrix units inside the GPU.
Nvidia's V100 added Tensor Cores, delivering 125 teraFLOPS for deep learning. Matrix engines became the core of AI hardware.
What it built on 2
What it led to 2
Related measurements
The dashed line marks when it first worked.
ML hardware FLOP/s per dollar Source: Epoch AI, ML hardware (CC BY 4.0)
Training compute of notable AI models Source: Epoch AI, Notable AI Models (CC BY 4.0)